RE: Class initialization with multiple inheritance

[Joseph L. Casale]

-Original Message-
From: Barry Scott  
Sent: Tuesday, July 16, 2019 11:53 AM
To: Joseph L. Casale 
Cc: python-list@python.org
Subject: Re: Class initialization with multiple inheritance

> And here is the MRO for LeftAndRight.
>
> >>> import m
> LeftAndRight.__init__()
> Left.__init__()
> Right.__init__()
> CommonBase.__init__()
> >>> m.LeftAndRight.__mro__
> (,
>  ,
>  ,
>  ,
>  )
> >>> 

Thank you DL and Barry, that was very helpful.

It seems that if Base1, 2, or 3 creates its own arguments for the common base
class, this becomes notable and this one case becomes the arguments preserved
and sent to the base regardless of the order of Base1, 2 or 3 inside 
DerivedClassName.

Now if all three of Base1, 2, or 3 produce their own arguments (discarding 
those intended
from DerivedClassName, only one is chosen regardless of the order, indicating 
some
deterministic approach on the name possibly.

Presumably a choice had to be made as to how to handle this case (though the 
runtime
could have thrown), it would be enlightening to know what logic is utilized in 
this case.

Thanks a lot!
jlc
-- 
https://mail.python.org/mailman/listinfo/python-list

How to execute shell command in Python program?

[Madhavan Bomidi]

Hi,

I have the following shell commands to run my executable created with gfortran:
-
$ pwd
/OPAC/opac31a/opac31
$ ls
extback.dat opacopac.f  result  terr.dat
input   opac.cfgpyopac3.py  solar.dat   wel.dat
$ ./opac <==
 
 *  Optical Properties of Aerosols and Clouds OPAC 3.1  *
 *  --  *
 *  *
 *  The contents of OPAC has been described in: *
 *  M. Hess, P. Koepke and I. Schult (1997):*
 *   "Optical Properties of Aerosols and Clouds:*
 *The software package OPAC"*
 *  submitted to Bull. Am. Meteor. Soc. *
 *  *
 *  last update: 14.11.97 M. Hess   *
 


  name of input file (without extension .inp), (max. 8 characters)
  default: opac.inp
opac <==
  input file input/opac.inp used
  READY.
Note: The following floating-point exceptions are signalling: IEEE_DENORMAL
---

Now my concern is the following commands (pointed by <==) to be run within 
the Python program. I tried to use the following lines in the Python program:

-
import subprocess
subprocess.call(['./opac'],shell=True)

Out[21]: 2

I am not able to know how I input the 'opac' (which is the input filename)

Please help me.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to execute shell command in Python program?

[Chris Narkiewicz via Python-list]

Madhavan Bomidi wrote:
> import subprocess
> subprocess.call(['./opac'],shell=True)

subprocess.call(['./opac', "my-input.inp"], shell=True)

The array takes command with a list of arguments. This
way you don't need to do space escaping and other
Jujitsu gimmicks.

If you want to feed the command from stdin, this becomes a bit
more complicated as you need to start a subprocess and use
PIPE to feed it.

Cheers,
Chris Narkiewicz



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to execute shell command in Python program?

[Peter J. Holzer]

On 2019-07-20 15:39:58 +0100, Chris Narkiewicz via Python-list wrote:
> Madhavan Bomidi wrote:
> > import subprocess
> > subprocess.call(['./opac'],shell=True)

There may be an os.chdir() missing here.

> subprocess.call(['./opac', "my-input.inp"], shell=True)

We don't know whether the OP's program accepts command line arguments.
But I agree that it probably does and then this is the best way.


> The array takes command with a list of arguments. This
> way you don't need to do space escaping and other
> Jujitsu gimmicks.

In this case you should set shell=False. Either invoke the command
directly with an argument vector (almost always preferrable), or invoke
the shell with a shell command. Don't mix them.

hp


-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Proper shebang for python3

[Manfred Lotz]

Hi there,
Pretty new to python I've got a question regarding the proper shebang
for Python 3.

I use 
   #!/usr/bin/python3

which works fine.

Today I saw 
   #!/usr/bin/python3 -tt

and was wondering what -tt means.

Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even
mention -t. 

python 2 man page mentions

   -t Issue a warning when a source file mixes tabs and spaces
   for indentation in a way that makes  it  depend  on  the worth
   of a tab expressed in spaces.  Issue an error when the option is
   given twice.

I guess that -t has the same meaning with python 3.7.3. 


My questions:

1. Is my guess correct?

2. Is it a bug that it is not mentioned? python3 --help doesn't mention
it either.


-- 
Manfred


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Sun, Jul 21, 2019 at 3:41 AM Manfred Lotz  wrote:
>
> Hi there,
> Pretty new to python I've got a question regarding the proper shebang
> for Python 3.
>
> I use
>#!/usr/bin/python3
>
> which works fine.
>
> Today I saw
>#!/usr/bin/python3 -tt
>
> and was wondering what -tt means.
>
> Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even
> mention -t.
>
> python 2 man page mentions
>
>-t Issue a warning when a source file mixes tabs and spaces
>for indentation in a way that makes  it  depend  on  the worth
>of a tab expressed in spaces.  Issue an error when the option is
>given twice.
>
> I guess that -t has the same meaning with python 3.7.3.

Far as I know, the -tt option to Python 3 is accepted for 2/3
compatibility, but does absolutely nothing. Having it on the shebang
probably means that it used to be /usr/bin/python -tt, and got
migrated to python3 without removing it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Manfred Lotz]

On Sun, 21 Jul 2019 03:44:24 +1000
Chris Angelico  wrote:

> On Sun, Jul 21, 2019 at 3:41 AM Manfred Lotz 
> wrote:
> >
> > Hi there,
> > Pretty new to python I've got a question regarding the proper
> > shebang for Python 3.
> >
> > I use
> >#!/usr/bin/python3
> >
> > which works fine.
> >
> > Today I saw
> >#!/usr/bin/python3 -tt
> >
> > and was wondering what -tt means.
> >
> > Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't
> > even mention -t.
> >
> > python 2 man page mentions
> >
> >-t Issue a warning when a source file mixes tabs and
> > spaces for indentation in a way that makes  it  depend  on  the
> > worth of a tab expressed in spaces.  Issue an error when the option
> > is given twice.
> >
> > I guess that -t has the same meaning with python 3.7.3.  
> 
> Far as I know, the -tt option to Python 3 is accepted for 2/3
> compatibility, but does absolutely nothing. Having it on the shebang
> probably means that it used to be /usr/bin/python -tt, and got
> migrated to python3 without removing it.
> 

This is a plausible explanation.

Thanks, Manfred




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Michael Speer]

You may want to use `#!/usr/bin/env python3` instead.

There is a concept in python called the virtual environment. This used to
be done with a tool called virtualenv in python2, and is now done mainly
through a venv module in python3.

A virtual environment goes into a directory of your choosing and will have
its own python3 executable, and pip3 executable, and when you add
dependencies, they are also placed into the directory structure under your
chosen directory.

When you do a `. /bin/activate` the included source will places
the virtual environment's bin/ folder at the beginning of your PATH
environment variable, making it the default python3 when you type it
without a full path.

This allows you to run scripts that need different, or even conflicting,
sets of dependencies without bothering with the underlying linux
distribution's python installation's modules.

If you use `#!/usr/bin/python3`, it will always use exactly the system
version that is installed, and the system's installed modules.

Your scripts will still default to the system installation if a virtual
environment is not activated. So you lose nothing by doing it this way, but
gain a little control from it.


On Sat, Jul 20, 2019 at 1:41 PM Manfred Lotz  wrote:

> Hi there,
> Pretty new to python I've got a question regarding the proper shebang
> for Python 3.
>
> I use
>#!/usr/bin/python3
>
> which works fine.
>
> Today I saw
>#!/usr/bin/python3 -tt
>
> and was wondering what -tt means.
>
> Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even
> mention -t.
>
> python 2 man page mentions
>
>-t Issue a warning when a source file mixes tabs and spaces
>for indentation in a way that makes  it  depend  on  the worth
>of a tab expressed in spaces.  Issue an error when the option is
>given twice.
>
> I guess that -t has the same meaning with python 3.7.3.
>
>
> My questions:
>
> 1. Is my guess correct?
>
> 2. Is it a bug that it is not mentioned? python3 --help doesn't mention
> it either.
>
>
> --
> Manfred
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Sun, Jul 21, 2019 at 4:13 AM Michael Speer  wrote:
>
> You may want to use `#!/usr/bin/env python3` instead.
>
> There is a concept in python called the virtual environment. This used to
> be done with a tool called virtualenv in python2, and is now done mainly
> through a venv module in python3.
>
> A virtual environment goes into a directory of your choosing and will have
> its own python3 executable, and pip3 executable, and when you add
> dependencies, they are also placed into the directory structure under your
> chosen directory.
>
> When you do a `. /bin/activate` the included source will places
> the virtual environment's bin/ folder at the beginning of your PATH
> environment variable, making it the default python3 when you type it
> without a full path.
>
> This allows you to run scripts that need different, or even conflicting,
> sets of dependencies without bothering with the underlying linux
> distribution's python installation's modules.
>
> If you use `#!/usr/bin/python3`, it will always use exactly the system
> version that is installed, and the system's installed modules.
>
> Your scripts will still default to the system installation if a virtual
> environment is not activated. So you lose nothing by doing it this way, but
> gain a little control from it.
>

ONLY if you actually want this script to behave differently inside a
venv. When you're setting a shebang on a script, you often do not want
that. You potentially LOSE a lot of control.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Embedding Python in C

[Stefan Behnel]

Jesse Ibarra schrieb am 20.07.19 um 04:12:
> Sorry, I am not understanding. Smalltlak VW 8.3 does not support Python.
> I can only call Pyhton code through C/Python API.

Ok, but that doesn't mean you need to write code that uses the C-API of
Python. All you need to do is:

1) Start up a CPython runtime from Smalltalk (see the embedding example I
posted) and make it import an extension module that you write (e.g. using
the "inittab" mechanism [1]).

2) Use Cython to implement this extension module to provide an interface
between your Smalltalk code and your Python code. Use the Smalltalk C-API
from your Cython code to call into Smalltalk and exchange data with it.

Now you can execute Python code inside of Python and make it call back and
forth into your Smalltalk code, through the interface module. And there is
no need to use the Python C-API for anything beyond step 1), which is about
5 lines of Python C-API code if you write it yourself. Everything else can
be implemented in Cython and Python.

Stefan


[1]
https://docs.python.org/3/extending/embedding.html?highlight=PyImport_appendinittab#extending-embedded-python

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 1:20 PM, Chris Angelico wrote:
> On Sun, Jul 21, 2019 at 4:13 AM Michael Speer  wrote:
>>
>> You may want to use `#!/usr/bin/env python3` instead.
>>
>> There is a concept in python called the virtual environment. This used to
>> be done with a tool called virtualenv in python2, and is now done mainly
>> through a venv module in python3.
>>
>> A virtual environment goes into a directory of your choosing and will have
>> its own python3 executable, and pip3 executable, and when you add
>> dependencies, they are also placed into the directory structure under your
>> chosen directory.
>>
>> When you do a `. /bin/activate` the included source will places
>> the virtual environment's bin/ folder at the beginning of your PATH
>> environment variable, making it the default python3 when you type it
>> without a full path.
>>
>> This allows you to run scripts that need different, or even conflicting,
>> sets of dependencies without bothering with the underlying linux
>> distribution's python installation's modules.
>>
>> If you use `#!/usr/bin/python3`, it will always use exactly the system
>> version that is installed, and the system's installed modules.
>>
>> Your scripts will still default to the system installation if a virtual
>> environment is not activated. So you lose nothing by doing it this way, but
>> gain a little control from it.
>>
> 
> ONLY if you actually want this script to behave differently inside a
> venv. When you're setting a shebang on a script, you often do not want
> that. You potentially LOSE a lot of control.
> 
> ChrisA

Disagree strongly.  The environment should always define where you want to find 
key
binaries, whether in a venv or not.  There are many use cases where you want to
override system binaries.  For example, you may be running on an older OS 
release
and want to point to a newer one installed elsewhere.  Similarly, the DevOps 
workflow
may demand particular configurations for the pipelines to run properly.

I have spent way too much time in my career trying to undo this kind of 
imperious
programming that thinks the coder knows best, when in the actual runtime 
environment,
they actually don't.  Most recently, I have been working to produce a 
configuration
for linuxbrew that can be installed by an unprivileged user at any location 
they like.
It is maddening to spend hours getting things working only to find out that 
some genius
decided that the only version of, say, perl or autoconf, is to be found 
exclusively in
/usr/bin.

So, no, do NOT encode the hard location - ever.  Always use env to discover the 
one that
the user has specified.  The only exception is /bin/sh which - for a variety of 
reasons -
can reliably counted upon.

We don't need to bikeshed this.  All we need is people who disagree with this 
view to spend
a year in software packaging, operations, deployment and DevOps ... then get 
back to us...

Grr..

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Manfred Lotz]

On Sat, 20 Jul 2019 14:11:21 -0400
Michael Speer  wrote:

> You may want to use `#!/usr/bin/env python3` instead.
> 

In my case it doesn't matter. However, I agree that your suggestion is
usually preferable.

> There is a concept in python called the virtual environment. This
> used to be done with a tool called virtualenv in python2, and is now
> done mainly through a venv module in python3.
> 
> A virtual environment goes into a directory of your choosing and will
> have its own python3 executable, and pip3 executable, and when you add
> dependencies, they are also placed into the directory structure under
> your chosen directory.
> 
> When you do a `. /bin/activate` the included source will
> places the virtual environment's bin/ folder at the beginning of your
> PATH environment variable, making it the default python3 when you
> type it without a full path.
> 
> This allows you to run scripts that need different, or even
> conflicting, sets of dependencies without bothering with the
> underlying linux distribution's python installation's modules.
> 

New to me. Interesting. 

> If you use `#!/usr/bin/python3`, it will always use exactly the system
> version that is installed, and the system's installed modules.
> 
> Your scripts will still default to the system installation if a
> virtual environment is not activated. So you lose nothing by doing it
> this way, but gain a little control from it.
> 

Ok, understood.


-- 
Manfred


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Peter J. Holzer]

On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote:
> So, no, do NOT encode the hard location - ever.  Always use env to
> discover the one that the user has specified.  The only exception is
> /bin/sh which - for a variety of reasons - can reliably counted upon.
> 
> We don't need to bikeshed this.  All we need is people who disagree
> with this view to spend a year in software packaging, operations,
> deployment and DevOps ... then get back to us...

After 25 years in software packaging, operations, deployment and DevOps
I disagree: A program should not behave differently because of a
different path, #!/usr/bin/env is a total no-no.

There is a nice way to achieve this: Just use the interpreter of the
virtual environment in the shebang.
(That requires rewriting the shebang during installation, but that's a
minor inconvenience)

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 2:56 PM, Peter J. Holzer wrote:
> On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote:
>> So, no, do NOT encode the hard location - ever.  Always use env to
>> discover the one that the user has specified.  The only exception is
>> /bin/sh which - for a variety of reasons - can reliably counted upon.
>>
>> We don't need to bikeshed this.  All we need is people who disagree
>> with this view to spend a year in software packaging, operations,
>> deployment and DevOps ... then get back to us...
>
> After 25 years in software packaging, operations, deployment and DevOps
> I disagree: A program should not behave differently because of a
> different path, #!/usr/bin/env is a total no-no.
>
> There is a nice way to achieve this: Just use the interpreter of the
> virtual environment in the shebang.
> (That requires rewriting the shebang during installation, but that's a
> minor inconvenience)
>
> hp
>



And what happens with programs that have no virtenv equivalent?
perl, go, ruby, awk, sed, grep, etc. have no simple way to
get installed virtual short of insisting the everything live in a
docker container or VM?

The fact is that most large compute environments are slow to upgrade
the OS.  That means core tools also lag considerably behind as well.
Being able to install newer versions along side the OS' own and then
use them by default is manifestly necessary.  That's why users have
the ability to modify $PATH to suit their own needs.  All /usr/bin/env
does is to tell the interpreter, "honor the intent of the spawning shell".
This shouldn't even be a question ... and it's why so much garbage continues
to live forever.  Having to constantly code around old systems versions of
tools with not other recourse is just crazy.

In actual fact, the very best way to write portable, reliable, and operable
systems of code is to divorce them entirely (or as a nearly as possible) for
the OS tools as you can.  That's why docker works so well.  That's why go
avoids dynamic linking.

In my case, that's why I compile my own version of
languages and daily use utilities to live outside the OS.  I get absolutely
predicable behavior irrespective of my distro and whatever backleveled cruft
it has laying around.   I _know_ every version of every important tool my code
will be using ... all by setting up $PATH properly and using /usr/bin/env in
my interpreters.

If you want really big fun, try going into an older CentOS or RedHat instances 
and, say,
upgrading system python to python3.  It's super fun.  Yes, in that case, you 
COULD
use a venv.  But there are tons of other tools for which this is not an option -
gcc, autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are invariably
backleveled on production OS instances.  My way fixes that problem.  Your way
forces me to code around it ... which is a way worse solution.

You may have 25 years at this but I have 40 - does that make me nearly twice
as right?  Arguments from authority are silly.

P.S. https://www.tundraware.com/TechnicalNotes/Divorce-Your-Linux-Admin/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Brian Oney via Python-list]

On Sat, 2019-07-20 at 15:26 -0500, Tim Daneliuk wrote:
> On 7/20/19 2:56 PM, Peter J. Holzer wrote:
> > On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote:
> > > So, no, do NOT encode the hard location - ever.  Always use env to
> > > discover the one that the user has specified.  The only exception is
> > > /bin/sh which - for a variety of reasons - can reliably counted upon.
> > > 
> > > We don't need to bikeshed this.  All we need is people who disagree
> > > with this view to spend a year in software packaging, operations,
> > > deployment and DevOps ... then get back to us...
> > 
> > After 25 years in software packaging, operations, deployment and DevOps
> > I disagree: A program should not behave differently because of a
> > different path, #!/usr/bin/env is a total no-no.
> > 
> > There is a nice way to achieve this: Just use the interpreter of the
> > virtual environment in the shebang.
> > (That requires rewriting the shebang during installation, but that's a
> > minor inconvenience)
> > 
> > hp
> > 
> 
> 
> And what happens with programs that have no virtenv equivalent?
> perl, go, ruby, awk, sed, grep, etc. have no simple way to
> get installed virtual short of insisting the everything live in a
> docker container or VM?
> 
> The fact is that most large compute environments are slow to upgrade
> the OS.  That means core tools also lag considerably behind as well.
> Being able to install newer versions along side the OS' own and then
> use them by default is manifestly necessary.  That's why users have
> the ability to modify $PATH to suit their own needs.  All /usr/bin/env
> does is to tell the interpreter, "honor the intent of the spawning shell".
> This shouldn't even be a question ... and it's why so much garbage continues
> to live forever.  Having to constantly code around old systems versions of
> tools with not other recourse is just crazy.
> 
> In actual fact, the very best way to write portable, reliable, and operable
> systems of code is to divorce them entirely (or as a nearly as possible) for
> the OS tools as you can.  That's why docker works so well.  That's why go
> avoids dynamic linking.
> 
> In my case, that's why I compile my own version of
> languages and daily use utilities to live outside the OS.  I get absolutely
> predicable behavior irrespective of my distro and whatever backleveled cruft
> it has laying around.   I _know_ every version of every important tool my code
> will be using ... all by setting up $PATH properly and using /usr/bin/env in
> my interpreters.
> 
> If you want really big fun, try going into an older CentOS or RedHat 
> instances and, say,
> upgrading system python to python3.  It's super fun.  Yes, in that case, you 
> COULD
> use a venv.  But there are tons of other tools for which this is not an 
> option -
> gcc, autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are 
> invariably
> backleveled on production OS instances.  My way fixes that problem.  Your way
> forces me to code around it ... which is a way worse solution.
> 
> You may have 25 years at this but I have 40 - does that make me nearly twice
> as right?  Arguments from authority are silly.
> 
> P.S. https://www.tundraware.com/TechnicalNotes/Divorce-Your-Linux-Admin/
Nice.

Emacs (well spacemacs) is my authority.

The key sequence 'SPC i !' inserts

#!/usr/bin/env python

as specified in the insert-shebang package 
https://github.com/psachin/insert-shebang

My limited experience tells me to expect the user (me) to know what
they're doing, hence env.

Why not make a compromise? What would be a potential pitfall of the
following spitbang?

#!python








-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to execute shell command in Python program?

[DL Neil]

On 21/07/19 5:07 AM, Peter J. Holzer wrote:

On 2019-07-20 15:39:58 +0100, Chris Narkiewicz via Python-list wrote:

Madhavan Bomidi wrote:

import subprocess
subprocess.call(['./opac'],shell=True)


There may be an os.chdir() missing here.


subprocess.call(['./opac', "my-input.inp"], shell=True)


We don't know whether the OP's program accepts command line arguments.
But I agree that it probably does and then this is the best way.

...


In this case you should set shell=False. Either invoke the command
directly with an argument vector (almost always preferrable), or invoke
the shell with a shell command. Don't mix them.



+1 for simplicity


subprocess.call() is now described as "older" (since v3.5). The manual 
says: <>>


Note that if the existing Fortran s/w puts its output to stdout (which I 
think ALL of 'mine' do), then you will need to "capture" that output 
from the subprocess (not necessary if the Fortran writes to a 
report-file on disk).


If you are using v3.7+, use:

subprocess.run( [ ...etc... ], capture_output=True )

Older versions 3.5+:

subprocess.run( [ ...etc... ], stdout=subprocess.PIPE )


run() returns a subprocess.CompletedProcess class.

To check that opac executed correctly, inspect the
CompletedProcess.returncode.

To access any "captured" output from opac, use CompletedProcess.stdout 
(similarly stderr, if execution was unsuccessful).


Be aware that the returned data may be string or bytes, and may have 
different line-separators from the one you/your Operating System 
normally uses!


Please study the first sections of subprocess — Subprocess management 
for details, (https://docs.python.org/3/library/subprocess.html), 
experiment, and come back to us with any questions...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk  wrote:
> So, no, do NOT encode the hard location - ever.  Always use env to discover 
> the one that
> the user has specified.  The only exception is /bin/sh which - for a variety 
> of reasons -
> can reliably counted upon.

A quick grep through my $PATH shows that there are a number of
executable Python scripts there, including add-apt-repository,
calibre, some lilypond stuff, trash-can management, samba-tool, iotop,
and youtube-dl. If I have a venv active with, say, Python 3.9, then
`/usr/bin/env python` is going to point to Python 3.9. What are the
odds that all those scripts will work with Python 3.9 with no
libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be
affected by a virtual environment, when typing "man youtube-dl"
wouldn't be?? Using env for everything is a terrible idea and one that
will basically make virtual environments useless.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 5:14 PM, Chris Angelico wrote:
> Using env for everything is a terrible idea and one that
> will basically make virtual environments useless.

Not if you manage them properly.

Everyone's mileage is different, but when I enter a venv, I ensure everything I 
do
there is packaged to work together.  I pip install things in there and ensure
that the tools I invoke will work in that environment.

In the example you cite, I would simply run the tools in question outside any
environment where I know they don't work.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Sun, Jul 21, 2019 at 8:56 AM Tim Daneliuk  wrote:
>
> On 7/20/19 5:14 PM, Chris Angelico wrote:
> > Using env for everything is a terrible idea and one that
> > will basically make virtual environments useless.
>
> Not if you manage them properly.
>
> Everyone's mileage is different, but when I enter a venv, I ensure everything 
> I do
> there is packaged to work together.  I pip install things in there and ensure
> that the tools I invoke will work in that environment.
>
> In the example you cite, I would simply run the tools in question outside any
> environment where I know they don't work.

Are you aware of every systemwide command that happens to be
implemented in Python, such that you won't execute any of them while
you have the venv active? The ones I mentioned are all installed into
/usr/local/bin or /usr/bin or some other directory on my main $PATH. I
only know they're Python scripts because I just ran "file
/usr/local/bin/*" and had it check them all. If they all reacted to
the activation of a venv, I would have to be extremely careful of
which commands I ever run while a venv is active. Which basically
means I would never activate a venv ever again - not worth it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 5:47 PM, Tim Daneliuk wrote:
> On 7/20/19 5:14 PM, Chris Angelico wrote:
>> Using env for everything is a terrible idea and one that
>> will basically make virtual environments useless.
> 
> Not if you manage them properly.
> 
> Everyone's mileage is different, but when I enter a venv, I ensure everything 
> I do
> there is packaged to work together.  I pip install things in there and ensure
> that the tools I invoke will work in that environment.
> 
> In the example you cite, I would simply run the tools in question outside any
> environment where I know they don't work.
> 

I guess I should clarify something:

1) I pip install everything locally inside my own $HOME for that set of things 
I need
   generally and will not be using in a venv.

2) I pip install everything I need in each venv, even if it already exists 
under 1) above.

I effectively treat venvs as fully self-contained environments to the degree 
possible.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Cameron Simpson]

On 21Jul2019 08:14, Chris Angelico  wrote:

On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk  wrote:

So, no, do NOT encode the hard location - ever.  Always use env to discover the 
one that
the user has specified.  The only exception is /bin/sh which - for a variety of 
reasons -
can reliably counted upon.


A quick grep through my $PATH shows that there are a number of
executable Python scripts there, including add-apt-repository,
calibre, some lilypond stuff, trash-can management, samba-tool, iotop,
and youtube-dl. If I have a venv active with, say, Python 3.9, then
`/usr/bin/env python` is going to point to Python 3.9. What are the
odds that all those scripts will work with Python 3.9 with no
libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be
affected by a virtual environment, when typing "man youtube-dl"
wouldn't be?? Using env for everything is a terrible idea and one that
will basically make virtual environments useless.


I'm with Tim Daneliuk. The environment matters and should be honoured 
except in extremely weird cases.


If you require a specific outcoming, set a specific environment. It is 
under your control. Control it.


For your specific example, "man youtube-dl" _is_ affected by the 
environment. It honours the $MANPATH variable.


For Peter J. Holzer, if we must play the "I've been doing this forever" 
game: I've been sysadmining etc longer than your 25 years and disagree 
with you. If your tools are strongly affected by version, run them with 
an environment that finds the right version.


Installers, particularly those run as root or in other low level setup 
situations, should have a WELL DEFINED environment.


If you have a tool that isn't portable, put it in a wrapper that coddles 
its behaviour.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson  wrote:
>
> On 21Jul2019 08:14, Chris Angelico  wrote:
> >On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk  wrote:
> >> So, no, do NOT encode the hard location - ever.  Always use env to 
> >> discover the one that
> >> the user has specified.  The only exception is /bin/sh which - for a 
> >> variety of reasons -
> >> can reliably counted upon.
> >
> >A quick grep through my $PATH shows that there are a number of
> >executable Python scripts there, including add-apt-repository,
> >calibre, some lilypond stuff, trash-can management, samba-tool, iotop,
> >and youtube-dl. If I have a venv active with, say, Python 3.9, then
> >`/usr/bin/env python` is going to point to Python 3.9. What are the
> >odds that all those scripts will work with Python 3.9 with no
> >libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be
> >affected by a virtual environment, when typing "man youtube-dl"
> >wouldn't be?? Using env for everything is a terrible idea and one that
> >will basically make virtual environments useless.
>
> I'm with Tim Daneliuk. The environment matters and should be honoured
> except in extremely weird cases.
>
> If you require a specific outcoming, set a specific environment. It is
> under your control. Control it.
>
> For your specific example, "man youtube-dl" _is_ affected by the
> environment. It honours the $MANPATH variable.
>
> For Peter J. Holzer, if we must play the "I've been doing this forever"
> game: I've been sysadmining etc longer than your 25 years and disagree
> with you. If your tools are strongly affected by version, run them with
> an environment that finds the right version.
>
> Installers, particularly those run as root or in other low level setup
> situations, should have a WELL DEFINED environment.
>
> If you have a tool that isn't portable, put it in a wrapper that coddles
> its behaviour.

So you mean that a tool that depends on running on a consistent
environment, it should use a shebang of "/usr/bin/python3.6" instead
of "/usr/bin/env python3"? Because, wow, that would be exactly what is
already happening on my system. Why use /usr/bin/env and then wrap
something around it to force the environment, when you could just set
a correct shebang so it properly defines its execution environment?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Namespaces: memory vs 'pollution'

[DL Neil]

How do you remember to from-import- 'everything' that is needed?


I have a 'utility module' which contains a bunch of classes which 
examine/check/log aspects of the execution environment. One of which is 
PythonEnvironment, another relates to the HostSystem (as examples). They 
are most-frequently used to ensure that an application runs within an 
appropriate environment, eg don't use Posix code on MS-Windows and don't 
call for v3.7 features from Python3.5.


Today, I 'grabbed' the module, imported the PythonEnvironment, and 
PyTested the application code's Python version - as much for my 
amusement as anything else I asked for an impossible versionNR. Of 
course, the test failed.


Upon closer inspection, I realised it didn't just fail; it failed badly! 
Some silly, little, boy had imported the PythonEnvironment class but 
failed to ALSO import PythonVersionError. So, the reported error was not 
the expected exception!


Yes, the two classes appear in the module one-above-the-other, and yes, 
in the module's own pytest import-ing both classes is 'right there' in 
front of my nose. So, not my finest hour!



Is there some 'easy way' to make sure that one doesn't just import the 
desired class, but also remembers to import 'everything else' that might 
be necessary. In this case, it'd be rather nice to:


from environment_module import Python*

NB this is a syntax error, and won't import both the PythonEnvironment 
and PythonVersionError classes.


NBB I could import the module, fish-around in its __dict__, and do 
something like update-ing locals() with every key containing "Python" - 
but that is surely a truly, ugly, hack.



The 'lazy' answer is (purists, look away now!):

from environment_module import *

but that import-s "everything, including the kitchen sink"!

After you've recovered from the shock-horror of thinking about such 
blatant "namespace pollution", if the module only contains a bunch of 
classes, is it really such an evil act?
(and if the module's import-as-abbreviations are moved 'inside' the 
relevant class, those 'disappear' from the global namespace reducing the 
risk of name-clashes down to the names of the custom-classes and the PSL 
imports, eg os and platform)



What do you do to (respecting purism) ensure 'everything' (necessary) is 
imported (and nothing more), preferably without relying upon (faulty, in 
my case) human-memory or reading through volumes of code/documentation?


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Cameron Simpson]

On 21Jul2019 09:31, Chris Angelico  wrote:

On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson  wrote:
So you mean that a tool that depends on running on a consistent
environment, it should use a shebang of "/usr/bin/python3.6" instead
of "/usr/bin/env python3"?


Jeez. No. That is the _opposite_ of what I'm saying.


Because, wow, that would be exactly what is
already happening on my system. Why use /usr/bin/env and then wrap
something around it to force the environment, when you could just set
a correct shebang so it properly defines its execution environment?


Because the shebang is hardwired and inflexible.

Because it means hand patching (even scripted) a bazillion scripts to 
that they know their physical install.


Because it presumes detailed hardwired knowledge of the target system in 
a script which should work anywhere.


Instead a tiny _common_ shell script resembling this:

 #!/bin/sh
 # Run command in the official environment.
 exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"}

arranges things. The "env -" is aimed at "clean" daemon or install 
environments.  You can do subtler or less intrusive things in other 
settings.


There are any number of variations available here that don't hardwire 
the environment into the various _scripts_, but into the invocation 
setting. Or just start an environment (eg a shell) with the desired 
environment.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

How to print out html tags excluding the attributes

[sum abiut]

I want to use regular expression to print out the HTML tags excluding the
attributes.

for example:

import re
html = 'Hitest test'
tags = re.findall(r'<[^>]+>', html)
for a in tags:
print(a)


the output is :








But I just want the tag, not the attributes







-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to print out html tags excluding the attributes

[Chris Angelico]

On Sun, Jul 21, 2019 at 11:06 AM sum abiut  wrote:
>
> I want to use regular expression to print out the HTML tags excluding the
> attributes.

I'll just leave this here...

https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags

No, I won't be that cruel. I'll also suggest that Beautiful Soup is an
excellent non-regex way to parse HTML.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

SCons 3.1.0 Released

[Bill Deegan]

A new SCons checkpoint release, 3.1.0, is now available
  on the SCons download page:

  https://scons.org/pages/download.html

SCons is an Open Source software construction tool—that is, a
next-generation build tool. Think of SCons as an improved, cross-platform
substitute for the classic Make utility with integrated functionality
similar to autoconf/automake and compiler caches such as ccache. In short,
SCons is an easier, more reliable and faster way to build software.

  Here is a summary of the changes since 3.0.5:

  NEW FUNCTIONALITY

- Added variable TEMPFILEARGJOIN to specify how to join arguments
written
  to temp files used when command lines exceed MAXLINELENGTH when the
  command uses $TEMPFILE{...}
- Support for MSVC 2019
- Upgraded and improved Visual Studio solution/project generation code
using the MSVSProject builder.
  - Added support for Visual Studio 2017 and 2019.
  - Added support for the following per-variant parameters to the
builder:
- cpppaths: Provides per-variant include paths.
- cppdefines: Provides per-variant preprocessor definitions.


  CHANGED/ENHANCED EXISTING FUNCTIONALITY

  - Fix performance degradation for MD5-timestamp decider.  NOTE: This
changes the Decider() function arguments.
  From:
  def my_decider(dependency, target, prev_ni):
  To:
  def my_decider(dependency, target, prev_ni, repo_node):
  Where repo_node is the repository (or other) node to use to check if
the node is out of date instead of dependency.

- Enhanced --debug=explain output. Now the separate components of the
dependency list are split up
  as follows:

  scons: rebuilding `file3' because:
   the dependency order changed:
   ->Sources
   Old:xxx New:zzz
   Old:yyy New:yyy
   Old:zzz New:xxx
   ->Depends
   ->Implicit
   Old:/usr/bin/python New:/usr/bin/python

- Changed: Pseudo-builders now inherit OverrideEnvironments. For
  example when calling a pseudo-builder from another
  pseudo-builder the override variables passed to the first
  pseudo-builder call had to be explicitly passed on to the
  internal pseudo-builder call. Now the second pseudo-builder call
  will automatically inherit these override values.

  FIXES
- Fix Issue #3350 - SCons Exception EnvironmentError is conflicting
with Python's EnvironmentError.
- Fix spurious rebuilds on second build for cases where builder has > 1
target and the source file
  is generated. This was causing the > 1th target to not have it's
implicit list cleared when the source
  file was actually built, leaving an implicit list similar to follows
for 2nd and higher target
  ['/usr/bin/python', 'xxx', 'yyy', 'zzz']
  This was getting persisted to SConsign and on rebuild it would be
corrected to be similar to this
  ['zzz', 'yyy', 'xxx', '/usr/bin/python']
  Which would trigger a rebuild because the order changed.
  The fix involved added logic to mark all shared targets as peers and
then ensure they're implicit
  list is all cleared together.
- Fix Issue #3349 - SCons Exception EnvironmentError is conflicting
with Python's EnvironmentError.
  Renamed to SConsEnvironmentError
- Fix Issue #3350 - mslink failing when too many objects.  This is
resolved by adding TEMPFILEARGJOIN variable
  which specifies what character to join all the argements output into
the tempfile. The default remains a space
  when mslink, msvc, or mslib tools are loaded they change the
TEMPFILEARGJOIN to be a line separator (\r\n on win32)
- Additional fix to issue #3135 - Also handle 'pure' and 'elemental'
type bound procedures

- Fix handling of Visual Studio Compilers to properly reject any
unknown HOST_PLATFORM or TARGET_PLATFORM
- Enable LaTeX scanner to find more than one include per line

  under which they would be observed), or major code cleanups

git shortlog --no-merges -ns 3.0.5..HEAD
64  William Deegan
56  Mats Wichmann
10  Adam Gross
 4  Mathew Robinson
 4  Peter Diener
 3  Lukas Schrangl
 1  Daniel Holth
 1  bdbaddog
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Manfred Lotz]

On Sat, 20 Jul 2019 23:28:35 +0200
Brian Oney  wrote:

> Why not make a compromise? What would be a potential pitfall of the
> following spitbang?
> 
> #!python

I think that per definition a path in a shebang has to be absolute.

Actually, your suggestion won't work for people who use the fish
shell (just for the given reason).


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 6:04 PM, Chris Angelico wrote:
> Are you aware of every systemwide command that happens to be
> implemented in Python, such that you won't execute any of them while
> you have the venv active?

No, but this has never been a problem because the newer versions of
python tend to be pretty good - within a major release tree - of being
backward compatible.  Certainly, if I were running, say, RedHat 4 with
a very new version of python in my path first, this could theoretically
be an issue.  I've just not run into it.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Tim Daneliuk]

On 7/20/19 6:04 PM, Cameron Simpson wrote:
> If you require a specific outcoming, set a specific environment. It is under 
> your control. Control it.

Exactly right.  I have just had the REALLY irritating experience of trying to 
bootstrap a
location insensitive version of linuxbrew that mostly works, but is crippled by 
brain damage
that insists that the best version of perl will always be found in /usr/bin.

I have been a hardware engineer (analog and digital), software implementor, 
systems designer,
devops person, and large scale (physical data center) platform engineer and 
this kind of
bad thinking just kills site reliability.

I stipulate that there are corner cases where Chris A. is correct - it is 
certainly
possible to clobber a system with incorrect /usr/bin/env findings.   But I'd 
argue
that there is a simple work around - run another terminal session (terminator, 
tmux,
screen, ... whatever floats your boat) that has $PATH set up to find things in 
/usr/bin
first.  Voila! Problem solved and everyone can use env the way they want to.

In some respects, this problem does get a little simpler when you docker-ize 
your
software distributions until ... you need newer versions of tools than even the 
latest
docker OS implementations support.  Then you're back to the same old noise ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper shebang for python3

[Chris Angelico]

On Sun, Jul 21, 2019 at 3:41 PM Tim Daneliuk  wrote:
>
> On 7/20/19 6:04 PM, Chris Angelico wrote:
> > Are you aware of every systemwide command that happens to be
> > implemented in Python, such that you won't execute any of them while
> > you have the venv active?
>
> No, but this has never been a problem because the newer versions of
> python tend to be pretty good - within a major release tree - of being
> backward compatible.  Certainly, if I were running, say, RedHat 4 with
> a very new version of python in my path first, this could theoretically
> be an issue.  I've just not run into it.
>

Many Python packages are incompatible with CPython 3.9 at the moment. MANY.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list