Re: python 2.7.12 on Linux behaving differently than on Windows

[Steven D'Aprano]

On Monday 05 December 2016 17:20, DFS wrote:

>>> Edit: I got it to work this way:
>>> column2="'R'"
>>>
>>> but that's bogus, and I don't want users to have to do that.
>>
>> (1) It's not bogus.
> 
> 
> It's extremely bogus.  It's discarding part of my input.

When you type 

name = 'Fred'

do you expect that the first character of the variable name is a quotation 
mark? No? Then why is it okay for Python to discard the quotation mark but not 
bash or some other shell?

Shells don't just repeat the characters you type, they interpret them. 
Characters like $ & * ? [ ] { } and others have special meaning to the shell, 
as do quotation marks.


[...]
>> (3) If my interpretation is correct, what *is* bogus is that that your
>> column names include quotes in them.
> 
> 
> Of course not.  I'm not sure I've ever seen quotes or apostrophes in a
> column name.

*shrug* It would be a pretty crazy thing to do, and I'm very glad to hear that 
my wild guess as to what was going on was wrong.

> And that's a pretty crazy interpretation that nothing in my post would
> lead you to.

That's the curse of knowledge speaking: you're familiar enough with SQL that 
you can no longer remember what it was like to be unfamiliar with it.

I'm not familiar with SQL syntax, and to me, the fact that SQL was complaining 
when the quotation marks were missing from the column name suggested to me that 
the quotes were part of the name. To me, the analogy that came to my mind was 
the similar-looking error in the shell:


steve@runes:~$ ls "'foo'"
'foo'
steve@runes:~$ ls 'foo'
ls: cannot access foo: No such file or directory

The quotation marks are part of the filename, and so they need to be protected 
from the shell or else you get an error quite similar to the one you got:

no such column: R


but there is (or so it appeared) a column 'R' (hence it looks like the quotes 
are part of the column name).

Anyway, I'm glad you have a solution that satisfies you.



-- 
Steven
"Ever since I learned about confirmation bias, I've been seeing 
it everywhere." - Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 04/12/2016 23:25, Steve D'Aprano wrote:

On Mon, 5 Dec 2016 09:19 am, BartC wrote:


Command parameters /do/ behave differently between Windows and Linux,
for example try writing *.* as that third parameter.

In Windows, it will print *.*.

In Linux, if you have 273 files in the current directory, if will print
the name of the first, and there will be /272 further command
parameters/, each the name of a file. (I couldn't believe this when I
found out; one of my directories recently had 3.4 million files in it, I
don't really want *.* expanded to 3.4m arguments. Here, the fix is again
to use double quotes: "*.*". But what if the user doesn't do that?)


If the user doesn't escape the wildcard, then the shell will expand it,
exactly as the user would expect.

I'm not sure why you were surprised by that. * is a shell wildcard. By using
a * you are explicitly telling the shell to expand it to any file that
matches.


I don't know what a shell is. To me, it's some sort of user interface to 
the OS. So if someone types:


  > X A B C

You would expect X to be launched, and be given arguments A, B and C. 
You wouldn't expect any of them to be expanded to some unknown number of 
arguments.


In the same way that in code, you don't expect X(A,B,C) to be expanded 
to X(A,B0,B1,B2,B3,B4,B5,., C) if B happens to specify a slice.




Did you think it was a regular character like 'a' and 'z'?


If one of the parameters was a regular expression, would you expect it 
to be expanded to the entire possible set of inputs that match the 
expression?



I think it boils down to what the user expects. Linux and Unix users tend to
be technically-minded folks who use the command line a lot and demand
powerful tools, and they expect that wildcards like * should be expanded.


Which is dumb. How does the shell know exactly what I want to do with 
*.* or f*.c or whatever? Perhaps I don't want the expanded list right 
now (all the 3.4 million elements); perhaps I want to apply the same 
pattern to several directories; perhaps I'm passing it on to another 
program; perhaps I'm going to be writing it as another script; perhaps I 
just want to print out the parameter list; perhaps I want to transform 
or validate the pattern first; maybe I need to verify an earlier 
parameter before applying the pattern or the way it's applied depends on 
earlier arguments...


The input is a PATTERN; I want to process it, and apply it, myself.

And it doesn't work anyway; suppose I write:

  >X A *.* C D

How does the program know when the expanded filenames of *.* end, and 
the two extra parameters start? Remember it doesn't know there were four 
parameters, all it seems is one linear stream of arguments. Any 
structure the input may have had is lost.


What happens here:

 >X *.a *.b *.c

It'll just get one big long list of files, not three separate sets.

As I said, it's dumb. And expecting users to stick quotes everywhere is 
not a solution, because that's not going to happen.



Windows treats the command line as an afterthought, and until a few years
ago you were limited to a DOS shell. Today, your options are not as
limited: there's Powershell, and bash for Windows.


At least Windows does it properly. It doesn't even chop the command line 
into different parameters, making it considerably more flexible. (Unless 
you have a program based on a C-style main(nargs,args) entry point where 
the C runtime will do this for you.)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Mon, Dec 5, 2016 at 10:42 PM, BartC  wrote:
> At least Windows does it properly. It doesn't even chop the command line
> into different parameters, making it considerably more flexible. (Unless you
> have a program based on a C-style main(nargs,args) entry point where the C
> runtime will do this for you.)

Yes, because there's no way that you can ever get security problems
from improperly parsing command-line arguments. That's why the
recommended way to create a subprocess is os.system(), not the Popen
calls that take a list of already-separated parameters. Right?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 12:23, Chris Angelico wrote:

On Mon, Dec 5, 2016 at 10:42 PM, BartC  wrote:

At least Windows does it properly. It doesn't even chop the command line
into different parameters, making it considerably more flexible. (Unless you
have a program based on a C-style main(nargs,args) entry point where the C
runtime will do this for you.)


Yes, because there's no way that you can ever get security problems
from improperly parsing command-line arguments.


And you will never get any problems if a program expects 3 parameters 
but instead gets some arbitrary number of arguments, perhaps thousands, 
if one happens to be *, including some that could coincide with some 
actual meaningful input that the program recognises.


 That's why the

recommended way to create a subprocess is os.system(), not the Popen
calls that take a list of already-separated parameters. Right?


And nothing will ever go wrong with incorrectly calling Popen that 
takes, if I counted them correctly, up to 14 different parameters?


BTW what does Popen() do when one argument is '*.*'? Will that get 
expanded to multiple extra arguments, and at what point will it be 
expanded?


(I tried to test it, but:

  import subprocess
  subprocess.Popen("python")

didn't work under Linux: 'No such file or directory'. It works under 
Windows but I wanted to see what it did with a parameter *.


Another difference.)

--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

When will they fix Python _dbm?

[clvanwall]

I have been a Perl programmer for 15+ years and decided to give Python a try.  
My platform is windows and I installed the latest 3.5.2. Next I decided to 
convert a perl program that uses a ndbm database since according to the doc on 
python, it should be able to work with it.  Needless to say, I get: dbm.error: 
db type is dbm.ndbm, but the module is not available
Searching on Python Bug Tracker shows _dbm missing back in 03-03-2012!  That's 
a long time for a bug to be left open.  
John Van Walleghen


Sent from my Galaxy Tab® A
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 1:11 AM, BartC  wrote:
>
> BTW what does Popen() do when one argument is '*.*'? Will that get expanded
> to multiple extra arguments, and at what point will it be expanded?

Nope. Popen is not a shell.

It sounds as if you want a nerfed shell. Go ahead! I'm sure one
exists. It'll frustrate you no end once you get used to a better
shell, though - always does when I find myself on Windows...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Paul Moore]

On Monday, 5 December 2016 14:11:34 UTC, BartC  wrote:
> On 05/12/2016 12:23, Chris Angelico wrote:
> > On Mon, Dec 5, 2016 at 10:42 PM, BartC  wrote:
> >> At least Windows does it properly. It doesn't even chop the command line
> >> into different parameters, making it considerably more flexible. (Unless 
> >> you
> >> have a program based on a C-style main(nargs,args) entry point where the C
> >> runtime will do this for you.)
> >
> > Yes, because there's no way that you can ever get security problems
> > from improperly parsing command-line arguments.
> 
> And you will never get any problems if a program expects 3 parameters 
> but instead gets some arbitrary number of arguments, perhaps thousands, 
> if one happens to be *, including some that could coincide with some 
> actual meaningful input that the program recognises.

Windows and Linux are different. Neither is unambiguously "right" nor is either 
unambiguously "wrong". In both cases you need to understand what happens when 
you type a command, or you *will* get caught out by corner cases.

Calling either approach "dumb" is neither helpful nor productive.

For this specific example, of a program that takes a fragment of SQL as its 
command line, is one that's very hard to handle cleanly in a cross-platform 
way, because you actually don't want the shell, or the application, to 
interpret your arguments for you. The "best" approach is often to accept the 
SQL command as a single argument (argv[1]) and rely on your users quoting the 
argument appropriately. Admittedly, that simply pushes the problem onto your 
users, who may well also be uncomfortable with the subtleties of command line 
parsing, but at least they are using their choice of shell, so they have a 
chance of knowing.

The alternative, if you *really* don't want to force your users to understand 
shell parsing, is to prompt the user for the SQL - either as a simple console 
input, or (for users who are really uncomfortable with the command line) via a 
GUI program and a dialog box.

But criticising the (entirely valid, simply different) choices of another OS 
vendor as "dumb" isn't acceptable, nor is it a way to get to a solution to your 
issue.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Paul Moore]

On Monday, 5 December 2016 15:06:05 UTC, Chris Angelico  wrote:
> On Tue, Dec 6, 2016 at 1:11 AM, BartC  wrote:
> >
> > BTW what does Popen() do when one argument is '*.*'? Will that get expanded
> > to multiple extra arguments, and at what point will it be expanded?
> 
> Nope. Popen is not a shell.
> 
> It sounds as if you want a nerfed shell. Go ahead! I'm sure one
> exists. It'll frustrate you no end once you get used to a better
> shell, though - always does when I find myself on Windows...
> 
> ChrisA

For a non-nerfed (but *radically* different to bash) Windows shell, try 
Powershell. You'll probably hate it, but not because it's limited in 
capabilities :-)

Paul

PS Apparently, powershell is now open source and available for Linux and OSX. 
See https://github.com/PowerShell/PowerShell - although I don't know if all the 
features available on Windows exist on other platforms (yet).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 2:17 AM, Paul  Moore  wrote:
> For a non-nerfed (but *radically* different to bash) Windows shell, try 
> Powershell. You'll probably hate it, but not because it's limited in 
> capabilities :-)
>

Radically different from every shell I've ever called a shell. It
looks and feels more like a scripting language than a shell. Here's a
Windows batch script I put together that uses PS to do some of its
work:

https://github.com/Rosuav/Gypsum/blob/master/get_gypsum.bat

Granted, that was put together simply by searching Stack Overflow for
"how do I do *this* in PowerShell", so there might be other ways to do
it; but it feels to me like a VBScript interpreter, not a shell.

But hey. It exists by default on recent-enough Windowses, has the
features I want, and can be invoked by someone double-clicking on a
batch file. If you call it "flurgle", it won't make any difference to
me or to my work.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 15:05, Chris Angelico wrote:

On Tue, Dec 6, 2016 at 1:11 AM, BartC  wrote:


BTW what does Popen() do when one argument is '*.*'? Will that get expanded
to multiple extra arguments, and at what point will it be expanded?


Nope. Popen is not a shell.

It sounds as if you want a nerfed shell. Go ahead! I'm sure one
exists. It'll frustrate you no end once you get used to a better
shell, though - always does when I find myself on Windows...


That's not the point I was making.

Say you have this program a.py:

  import sys
  print (sys.argv)

And let's say there are just 3 files in the current directory: a.py, 
b.py and c.py.


If run from a Linux shell:

  python a.py *

The output is: ['a.py', 'b.py', 'c.py'] or something along those lines 
(there might be two copies of a.py).


Are you saying that if someone executes:

  subprocess.Popen(["python","a.py", "*"])

the output will be: ['a.py','*']?

In that case forget Windows vs. Linux, you now have a program that will 
get command parameters processed differently depending on whether it was 
invoked from a shell or not.


Or a program that sometimes will see "*" as an argument, and sometimes a 
big list of files that merges into all the other arguments.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: When will they fix Python _dbm?

[Anton Mamaenko]

What is your operating system, environment, and Python build? 

dbm is just a module that might not have been included into your Python build. 
It's not a bug but a deliberate choice of the package maker. 

Regards,
   Anton

> On 5 Dec 2016, at 17:45, clvanwall  wrote:
> 
> I have been a Perl programmer for 15+ years and decided to give Python a try. 
>  My platform is windows and I installed the latest 3.5.2. Next I decided to 
> convert a perl program that uses a ndbm database since according to the doc 
> on python, it should be able to work with it.  Needless to say, I get: 
> dbm.error: db type is dbm.ndbm, but the module is not available
> Searching on Python Bug Tracker shows _dbm missing back in 03-03-2012!  
> That's a long time for a bug to be left open.  
> John Van Walleghen
> 
> 
> Sent from my Galaxy Tab® A
> -- 
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 2:41 AM, BartC  wrote:
>
> Are you saying that if someone executes:
>
>   subprocess.Popen(["python","a.py", "*"])
>
> the output will be: ['a.py','*']?
>
> In that case forget Windows vs. Linux, you now have a program that will get
> command parameters processed differently depending on whether it was invoked
> from a shell or not.

Yes, that is correct. *SHELLS DO STUFF*. If you can't comprehend this,
you should get to know your shell. Try this:

python a.py %PATH%

subprocess.Popen(["python", "a.py", "%PATH%"])

Would you expect those to do the same? If you do, prepare for Windows
to surprise you. If you don't, why do you keep expecting other shells
to do nothing?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Paul Moore]

On Monday, 5 December 2016 15:41:59 UTC, BartC  wrote:
> On 05/12/2016 15:05, Chris Angelico wrote:
> > On Tue, Dec 6, 2016 at 1:11 AM, BartC  wrote:
> >>
> >> BTW what does Popen() do when one argument is '*.*'? Will that get expanded
> >> to multiple extra arguments, and at what point will it be expanded?
> >
> > Nope. Popen is not a shell.
> >
> > It sounds as if you want a nerfed shell. Go ahead! I'm sure one
> > exists. It'll frustrate you no end once you get used to a better
> > shell, though - always does when I find myself on Windows...
> 
> That's not the point I was making.
> 
> Say you have this program a.py:
> 
>import sys
>print (sys.argv)
> 
> And let's say there are just 3 files in the current directory: a.py, 
> b.py and c.py.
> 
> If run from a Linux shell:
> 
>python a.py *
> 
> The output is: ['a.py', 'b.py', 'c.py'] or something along those lines 
> (there might be two copies of a.py).
> 
> Are you saying that if someone executes:
> 
>subprocess.Popen(["python","a.py", "*"])
> 
> the output will be: ['a.py','*']?
> 
> In that case forget Windows vs. Linux, you now have a program that will 
> get command parameters processed differently depending on whether it was 
> invoked from a shell or not.
> 
> Or a program that sometimes will see "*" as an argument, and sometimes a 
> big list of files that merges into all the other arguments.


Python programs, when started, get a list of arguments via sys.argv.

1. On Linux, the OS primitive for executing a program takes a list of arguments 
and passes them directly. The user's shell is responsible for splitting a 
command line into "arguments".
2. On Windows, the OS primitive takes a command line. The application is 
responsible for splitting it into arguments, if it wants to. Most do, for 
compatibility with the normal argv convention inherited via C from Unix. Many 
programs let the C runtime do that splitting for them - I don't recall if 
Python does, or if it implements its own splitting these days.
3. The Popen class directly passes the supplied argv to the called program. 
(Technically, it has to do some nasty internal juggling to preserve the argv, 
but you don't need to care about that).

The program always gets a list of arguments. What *provides* that list to it 
(the Unix shell, the C/Python runtime, or the caller of Popen) varies. And you 
need (in more complex cases) to understand how the calling environment 
constructs that list of arguments if you want to reason about behaviour.

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 15:53, Chris Angelico wrote:

On Tue, Dec 6, 2016 at 2:41 AM, BartC  wrote:


Are you saying that if someone executes:

  subprocess.Popen(["python","a.py", "*"])

the output will be: ['a.py','*']?

In that case forget Windows vs. Linux, you now have a program that will get
command parameters processed differently depending on whether it was invoked
from a shell or not.


Yes, that is correct. *SHELLS DO STUFF*. If you can't comprehend this,
you should get to know your shell. Try this:

python a.py %PATH%

subprocess.Popen(["python", "a.py", "%PATH%"])

Would you expect those to do the same? If you do, prepare for Windows
to surprise you. If you don't, why do you keep expecting other shells
to do nothing?


You still don't get point. I write a program P, a native executable. It 
takes command line parameters but exactly what it gets depends on 
whether it's started from a 'shell' or from inside another program? I 
don't want to worry about that stuff or exactly how it is invoked!


> subprocess.Popen(["python", "a.py", "%PATH%"])

Yes, %...% is an escape sequence. Those % signs are supposed to stand 
out and have been chosen not to clash with typical input.


And the end result of the transformation is, here, also a SINGLE thing; 
any parameter following will still be the second parameter, not the 14771th!


Are you saying that the * in ABC*EF also makes the whole thing some 
escape pattern? And one that could massively expand the number of 
parameters, pushing all the following ones out of the way, and making it 
impossible to discover where these expanded parameters end and the 
normal ones recommence.


If someone had thought this up now, it would rapidly be dismissed as 
being unworkable. But because it's been in Unix/Linux/whatever so long, 
no one can see anything wrong with it!


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 3:23 AM, BartC  wrote:
> You still don't get point. I write a program P, a native executable. It
> takes command line parameters but exactly what it gets depends on whether
> it's started from a 'shell' or from inside another program? I don't want to
> worry about that stuff or exactly how it is invoked!
>
>> subprocess.Popen(["python", "a.py", "%PATH%"])
>
> Yes, %...% is an escape sequence. Those % signs are supposed to stand out
> and have been chosen not to clash with typical input.
>
> And the end result of the transformation is, here, also a SINGLE thing; any
> parameter following will still be the second parameter, not the 14771th!
>
> Are you saying that the * in ABC*EF also makes the whole thing some escape
> pattern? And one that could massively expand the number of parameters,
> pushing all the following ones out of the way, and making it impossible to
> discover where these expanded parameters end and the normal ones recommence.

Yes. That is exactly what I am saying. Also, try this:

set ENV_VAR=test1 test2
python a.py arg1 %ENV_VAR% arg2

How many args do you get? Is the end result really a single thing? Is
that single-thing-ness consistent across applications?

Windows's cmd.exe is not as simple as you think it is. Linux's bash is
not as insane as you think it is. Both of them take user input,
*MANIPULATE IT*, and pass it along. Keep on making these assumptions
and we will keep on proving them false.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Steve D'Aprano]

On Mon, 5 Dec 2016 10:42 pm, BartC wrote:

> I don't know what a shell is. To me, it's some sort of user interface to
> the OS. 

https://en.wikipedia.org/wiki/Unix_shell

You've never used cmd.com or command.exe? "The DOS prompt"? That's
(effectively) a shell.

Pedants may wish to explain exactly why the DOS prompt isn't a shell but to
a first approximation I think its close enough.

And yes, that's exactly what it is: its a text-based user interface to the
OS. And like any user-interface, designers can choose different user
interfaces, which will vary in power and convenience. And in the Unix/Linux
world, the command shell is not just a text interface, its a powerful
command interpreter and programming language.



> So if someone types: 
> 
>> X A B C
> 
> You would expect X to be launched, and be given arguments A, B and C.

Would I? I don't think so.

Even the DOS prompt supports some level of globbing. Its been a while since
I've used the DOS prompt in anger, but I seem to recall being able to do
things like:

dir a*

to get a listing of all the files starting with "a". So *something* is
treating the * as a special character. In Linux and Unix, that's invariably
the shell, before the dir command even sees what you typed.

In DOS, it might be the dir command itself. The disadvantage of the DOS way
of doing this is that *every single command and application* has to
re-implement its own globbing, very possibly inconsistently. That's a lot
of duplicated work and re-inventing the wheel, and the user will never know
what 

some_program a*

will do. Will it operate on all the files in the current directory that
start with "a"? Or will it try to operate on a single file called
literally "a*"? Which of course won't exist because * is a reserved
character on DOS/Windows file systems. You can't know ahead of time unless
you study the manual to see what metacharacters this specific command
supports.

The Unix way is far more consistent: applications typically don't have to
care about globbing, because the shell handles glob expansion, environment
variables, etc.

[Aside: if you take the big picture, the Unix/Linux way is probably LESS
consistent, because you can have any number of shells (sh, ash, bash, csh,
tcsh, dash, hush, zsh, and I expect many more). But in practice, there's
one lowest-common-denominator standard (sh) and one major de facto standard
(bash), and most of the shells are supersets of the original sh, so simple
things like wildcards behave in pretty similar ways.]

The downside of this is that if you don't want metacharacters expanded, you
have to tell the shell to ignore it. The easiest way is to escape them with
a backslash, or quote the string. But of course this being Unix, its
completely configurable (using an obscure syntax that's different for every
shell):

http://stackoverflow.com/questions/11456403/stop-shell-wildcard-character-expansion



> You wouldn't expect any of them to be expanded to some unknown number of
> arguments.

Actually yes I would. If they could be interpreted as file names with
globbing or environment variables, that's exactly what I would expect.

Even at the DOS prompt.

And I'd be completely gobsmacked if (say) the dir command understood the ?
metacharacter but the cp command didn't.


> In the same way that in code, you don't expect X(A,B,C) to be expanded
> to X(A,B0,B1,B2,B3,B4,B5,., C) if B happens to specify a slice.

In Python? No, I wouldn't expect that. Python's not a shell, and the design
is different. In Python, you have to use the metacharacter * to expand a
single argument into multiple arguments.



>> Did you think it was a regular character like 'a' and 'z'?
> 
> If one of the parameters was a regular expression, would you expect it
> to be expanded to the entire possible set of inputs that match the
> expression?

No, because Unix shells use globs, not regexes. Just like the DOS prompt.
Globs are simpler and require less typing, something system administrators
appreciate because (unlike programmers) interactive commands are written
far more than they are read, so brevity is appreciated.

(That doesn't excuse the umount command though. Really, you couldn't include
the "n"?)

So yes, I would expect that if I said

dir a*

I would get a listing of all the files starting with "a", not just the
single file called literally "a*".



>> I think it boils down to what the user expects. Linux and Unix users tend
>> to be technically-minded folks who use the command line a lot and demand
>> powerful tools, and they expect that wildcards like * should be expanded.
> 
> Which is dumb. How does the shell know exactly what I want to do with
> *.* or f*.c or whatever? Perhaps I don't want the expanded list right 
> now (all the 3.4 million elements); 

Sure, no problem. Just escape the metacharacters so they aren't expanded,
just as in Python if you want the literal string backslash n you can
write "\\n" to escape the backslash metacharacter.


> perha

Re: python 2.7.12 on Linux behaving differently than on Windows

[Lew Pitcher]

On Monday December 5 2016 10:41, in comp.lang.python, "BartC" 
wrote:

> On 05/12/2016 15:05, Chris Angelico wrote:
>> On Tue, Dec 6, 2016 at 1:11 AM, BartC  wrote:
>>>
>>> BTW what does Popen() do when one argument is '*.*'? Will that get
>>> expanded to multiple extra arguments, and at what point will it be
>>> expanded?
>>
>> Nope. Popen is not a shell.
>>
>> It sounds as if you want a nerfed shell. Go ahead! I'm sure one
>> exists. It'll frustrate you no end once you get used to a better
>> shell, though - always does when I find myself on Windows...
> 
> That's not the point I was making.
> 
> Say you have this program a.py:
> 
>import sys
>print (sys.argv)
> 
> And let's say there are just 3 files in the current directory: a.py,
> b.py and c.py.
> 
> If run from a Linux shell:
> 
>python a.py *
> 
> The output is: ['a.py', 'b.py', 'c.py'] or something along those lines
> (there might be two copies of a.py).

And, that's because, before invoking Popen, the SHELL has globbed that '*'
argument into the three filenames and substituted those names where the '*'
was.

If you don't use a shell, then (on Unix), you have to perform the globbing
yourself before invoking Popen.

> 
> Are you saying that if someone executes:
> 
>subprocess.Popen(["python","a.py", "*"])
> 
> the output will be: ['a.py','*']?
> 
> In that case forget Windows vs. Linux, you now have a program that will
> get command parameters processed differently depending on whether it was
> invoked from a shell or not.

Yes.

> Or a program that sometimes will see "*" as an argument, and sometimes a
> big list of files that merges into all the other arguments.
> 

Yes.

-- 
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Marko Rauhamaa]

Chris Angelico :

> On Tue, Dec 6, 2016 at 2:17 AM, Paul  Moore  wrote:
>> For a non-nerfed (but *radically* different to bash) Windows shell,
>> try Powershell. You'll probably hate it, but not because it's limited
>> in capabilities :-)
>
> Radically different from every shell I've ever called a shell. It
> looks and feels more like a scripting language than a shell.

In recent years, I've been disillusioned with bash and started using
Python more and more where I would previously have used bash. Python's
explicit syntax does automatically give you a level of security, but I
must say the subprocess.Popen.communicate syntax is painful as hell.
Your traditional one-liners turn into five-liners, and a casual observer
will have a slightly hard time understanding what's going on.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 3:53 AM, Marko Rauhamaa  wrote:
> In recent years, I've been disillusioned with bash and started using
> Python more and more where I would previously have used bash. Python's
> explicit syntax does automatically give you a level of security, but I
> must say the subprocess.Popen.communicate syntax is painful as hell.
> Your traditional one-liners turn into five-liners, and a casual observer
> will have a slightly hard time understanding what's going on.

Congratulations. You've just discovered why bash is useful.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Skip Montanaro]

On Mon, Dec 5, 2016 at 10:49 AM, Steve D'Aprano
 wrote:
>
> In DOS, it might be the dir command itself. The disadvantage of the DOS way
> of doing this is that *every single command and application* has to
> re-implement its own globbing, very possibly inconsistently. That's a lot
> of duplicated work and re-inventing the wheel, and the user will never know
> what
>
> some_program a*
>
> will do.

ISTR that the way DOS/Windows operate at the text prompt level was
modeled on VMS. As you indicated, each command was responsible for its
own "globbing". I've never programmed in DOS or Windows, and its been
decades since I programmed in VMS, but I imagine that both
environments probably provide some standard sort of globbing library.

On an only peripherally related note, I was reminded this morning of
how some/many GUI environments try to protect people from themselves.
I am just now coming up to speed in a new job which provides me with a
Windows window onto an otherwise Linux development environment. I
tried starting the X server this morning (something called Xming), and
it complained about not being able to write its log file (I suspect
Xming was alread. I tried to navigate to that location through the
Computer doohickey (Explorer?) but couldn't get there. The program (or
more likely the program's programmers) had decided that I had no
business "exploring" into my AppData folder. To get there, I had to
drop into a command prompt.

So, another vote for a text/shell interface. It gives you enough rope
to hang yourself, but assumes you won't, because, "we're all adults
here." I do understand why Windows hides stuff from users in the GUI
though. As a webmas...@python.org monitor, I can attest to the
relatively large number of questions received there asking about
removing Python "because I don't use it for anything." :-) This
started happening about the time the long defunct Compaq started to
write admin tools for Windows in Python.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: When will they fix Python _dbm?

[justin walters]

On Mon, Dec 5, 2016 at 6:45 AM, clvanwall  wrote:

> I have been a Perl programmer for 15+ years and decided to give Python a
> try.  My platform is windows and I installed the latest 3.5.2. Next I
> decided to convert a perl program that uses a ndbm database since according
> to the doc on python, it should be able to work with it.  Needless to say,
> I get: dbm.error: db type is dbm.ndbm, but the module is not available
> Searching on Python Bug Tracker shows _dbm missing back in 03-03-2012!
> That's a long time for a bug to be left open.
> John Van Walleghen
>
>
> Sent from my Galaxy Tab® A
> --
> https://mail.python.org/mailman/listinfo/python-list
>


Hi there,

Could you please post code that is giving you an error?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Steve D'Aprano]

On Tue, 6 Dec 2016 02:41 am, BartC wrote:

> In that case forget Windows vs. Linux, you now have a program that will
> get command parameters processed differently depending on whether it was
> invoked from a shell or not.

Er, yeah? You say that as if it were a bad thing.

Look at it this way. Suppose you were a really cunning system administrator
who spent 8 or 10 hours a day typing in commands and arguments that were
frequently file names. So you built yourself a macro system where you type
a command:

foo bar baz

and the macro would walk through the command line, automatically expanding
environment variables and file name globs. And because you used this 95% of
the time, and it was intended as a convenience for interactive use, you
linked the macro to the Enter key, so that only a single key press was
needed to do this processing.

But of course you're not an idiot, you know full well that there are
occasions where you want to avoid the expansion. So you include commands to
disable and enable that macro, and to give you even more fine-grained
control of what is expanded where, you build it escaping mechanisms so that
you can expand part of the command line and not other parts.

And then I come along, full of righteous indignation, and start yelling

"Wait a minute, now your application gets its command parameters processed
differently depend on whether you called it from C or used your macro!"

And you would answer:

"Of course it does, that's the whole bloody point of the macro!"

(By the way, I think that you will find that when you call Popen, if you set
shell=True it will be invoked in a subshell, which means you'll get the
full shell experience including command expansion. For good and evil.
That's a security risk, if you're getting the arguments from an untrusted
source, so don't pass shell=True unless you know what you're doing.)


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Marko Rauhamaa]

Chris Angelico :

> On Tue, Dec 6, 2016 at 3:53 AM, Marko Rauhamaa  wrote:
>> In recent years, I've been disillusioned with bash and started using
>> Python more and more where I would previously have used bash.
>> Python's explicit syntax does automatically give you a level of
>> security, but I must say the subprocess.Popen.communicate syntax is
>> painful as hell. Your traditional one-liners turn into five-liners,
>> and a casual observer will have a slightly hard time understanding
>> what's going on.
>
> Congratulations. You've just discovered why bash is useful.

Bash is nice, too nice. It makes it easy to write code that's riddled
with security holes. The glorious Unix tradition is to ignore the
pitfalls and forge ahead come what may.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 4:23 AM, Marko Rauhamaa  wrote:
> Chris Angelico :
>
>> On Tue, Dec 6, 2016 at 3:53 AM, Marko Rauhamaa  wrote:
>>> In recent years, I've been disillusioned with bash and started using
>>> Python more and more where I would previously have used bash.
>>> Python's explicit syntax does automatically give you a level of
>>> security, but I must say the subprocess.Popen.communicate syntax is
>>> painful as hell. Your traditional one-liners turn into five-liners,
>>> and a casual observer will have a slightly hard time understanding
>>> what's going on.
>>
>> Congratulations. You've just discovered why bash is useful.
>
> Bash is nice, too nice. It makes it easy to write code that's riddled
> with security holes. The glorious Unix tradition is to ignore the
> pitfalls and forge ahead come what may.

Bash assumes that the person typing commands has the full power to
execute commands. I'm not sure what you mean by "security holes",
unless it's passing text through bash that came from people who aren't
allowed to type commands.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 16:49, Steve D'Aprano wrote:

On Mon, 5 Dec 2016 10:42 pm, BartC wrote:



So if someone types:

   > X A B C

You would expect X to be launched, and be given arguments A, B and C.


Would I? I don't think so.

Even the DOS prompt supports some level of globbing. Its been a while since
I've used the DOS prompt in anger, but I seem to recall being able to do
things like:

dir a*

to get a listing of all the files starting with "a". So *something* is
treating the * as a special character. In Linux and Unix, that's invariably
the shell, before the dir command even sees what you typed.

In DOS, it might be the dir command itself.


Yes, it has to be, as there is probably no space to first construct an 
in-memory list of all the files.


 The disadvantage of the DOS way

of doing this is that *every single command and application* has to
re-implement its own globbing, very possibly inconsistently. That's a lot
of duplicated work and re-inventing the wheel,


Which will need to be done anyway. Expansion of filespecs with wildcards 
may need to be done from inside a program.


On Windows that involves calling FindFirstFile/FindNextFile (which takes 
care of wildcards for you), and on Linux it might be opendir/readdir 
(which doesn't; you have to call fnmatch to accept/reject each file).


(I had to port such code recently across OSes for my language; on both 
systems, dirlist(filespec) returns a list of files matching the wildcard 
specification provided. No shell expansion is needed!)



The Unix way is far more consistent: applications typically don't have to
care about globbing, because the shell handles glob expansion, environment
variables, etc.


It's not consistent because program P will sometimes see * and sometimes 
a list of files. On Windows P will /never/ see a list of files if the 
start point is *. Not without a lot of intervention.


And I've already posted a long list of reasons why Linux shell's 
behaviour is undesirable, as you want to literal argument, or you want 
to do something with the filespec that is different from what the shell 
will do, or you want to do it later (perhaps the files question DON'T 
EXIST until halfway through the program).


OK I'm just thinking up more reasons why I don't like Linux shell's 
behaviour.



You wouldn't expect any of them to be expanded to some unknown number of
arguments.


Actually yes I would. If they could be interpreted as file names with
globbing or environment variables, that's exactly what I would expect.


If the syntax is:

  program filespec

or:

  program filespec file

how do you tell whether the last file in an argument list is the 
optional 'file', or the last file of the expansion of 'filespec'?



So yes, I would expect that if I said

dir a*

I would get a listing of all the files starting with "a", not just the
single file called literally "a*".


So what does 'dir' do then; just print?

Since it sounds like:

  echo *.*

would do the job just as well! (If 'echo' is a program that just lists 
its input.)



Fine. Just escape the damn thing and do whatever you like to it.


I'm not the one running the program. Perhaps you don't know how stupid 
users can be.



The input is a PATTERN; I want to process it, and apply it, myself.


When you double-click on a .doc file,


Are we still talking about a console or terminal here? Any filenames 
displayed as part of the dialogue (result of ls or dir for example) are 
usually not clickable.



and Windows launches Word and opens
the file for editing, do you rant and yell that you didn't want Word to
open the file, you wanted to process the file name yourself?


GUIs are different.

--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 5:02 AM, BartC  wrote:
> If the syntax is:
>
>   program filespec
>
> or:
>
>   program filespec file
>
> how do you tell whether the last file in an argument list is the optional
> 'file', or the last file of the expansion of 'filespec'?

Why should you care? I have used shell globbing to pass precisely two
parameters to a program. More often, I use this syntax, which Windows
simply doesn't support:

ffmpeg -i some-really-long-file-name.{avi,mkv}

to convert a file from one format to another. And if the .mkv file
already exists, I can just use ".*" at the end, although that does
depend on them being in alphabetical order. The shell is predictable
and therefore useful. This trick is guaranteed to work, no matter what
program I'm using. On Windows, I have to hope that the program expands
these notations correctly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[eryk sun]

On Mon, Dec 5, 2016 at 3:41 PM, BartC  wrote:
>
> Are you saying that if someone executes:
>
>   subprocess.Popen(["python","a.py", "*"])
>
> the output will be: ['a.py','*']?
>
> In that case forget Windows vs. Linux, you now have a program that will get
> command parameters processed differently depending on whether it was invoked
> from a shell or not.
>
> Or a program that sometimes will see "*" as an argument, and sometimes a big
> list of files that merges into all the other arguments.

If it sees "*", it will try to open a file named "*". That's a valid
filename in Unix, but it should be avoided. We wouldn't want someone
to accidentally run `rm *` instead of `rm \*`.

In Windows, it's invalid for filenames to include wildcard characters
(i.e. '*' and '?' as well as the MS-DOS compatibility wildcards '<',
'>', and '"' -- DOS_STAR, DOS_QM, and DOS_DOT). Since there's no
ambiguity of intent, if you've linked an executable with
[w]setargv.obj, the C runtime will happily expand "*" in the argv
list.

I don't understand your concern regarding Unix shell scripting syntax.
On Windows, are you as troubled that environment variables such as
%USERNAME% get expanded by cmd.exe, but not by CreateProcess? Does it
bother you that cmd consumes its "^" escape character if it's not
escaped as "^^"? For example:

C:\>python.exe -i -c "" remove: ^ keep: ^^
>>> import win32api
>>> win32api.GetCommandLine()
'python.exe  -i -c "" remove:  keep: ^'

Every command-line shell that I've ever used is also a quirky
scripting language. Shell literacy requires learning at least the
syntax and operators of the language.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Printing a generator returns "", need to print its values

[vmahajan]

On Wednesday, November 16, 2016 at 3:25:39 PM UTC-5, Peter Otten wrote:
> vmaha...@centerpointmedia.com wrote:
> 
> > I am running Python2.7, wherein I am running the following price of code:
> > 
> > y = m.predict(input_fn=lambda:input_fn(df_predict), as_iterable=True)
> > print ('Predictions: {}'.format(str(y)))
> > 
> > The output of the following is "" > 0x7f0476373e10>"
> > 
> > However, the desired output must be in the format [1 0 0 1 1 0 0 1].
> > 
> > Can someone help me print the output in this format
> 
> You give no context, but my suggestion would be to try and omit the 
> 
> as_iterable=True
> 
> part...


Can someone help me print a generator object?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Gene Heskett]

On Monday 05 December 2016 12:23:31 Marko Rauhamaa wrote:

> Chris Angelico :
> > On Tue, Dec 6, 2016 at 3:53 AM, Marko Rauhamaa  
wrote:
> >> In recent years, I've been disillusioned with bash and started
> >> using Python more and more where I would previously have used bash.
> >> Python's explicit syntax does automatically give you a level of
> >> security, but I must say the subprocess.Popen.communicate syntax is
> >> painful as hell. Your traditional one-liners turn into five-liners,
> >> and a casual observer will have a slightly hard time understanding
> >> what's going on.
> >
> > Congratulations. You've just discovered why bash is useful.
>
> Bash is nice, too nice. It makes it easy to write code that's riddled
> with security holes. The glorious Unix tradition is to ignore the
> pitfalls and forge ahead come what may.
>
>
> Marko

I like bash, use it a lot.  But the ultimate language (damn the security 
considerations real or imagined) is still ARexx. The *nixes have not, 20 
years later, a do it all language that can play on the same field with 
ARexx.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 17:46, Dennis Lee Bieber wrote:

On Mon, 5 Dec 2016 11:42:08 +, BartC  declaimed the
following:



And it doesn't work anyway; suppose I write:

  >X A *.* C D

How does the program know when the expanded filenames of *.* end, and
the two extra parameters start? Remember it doesn't know there were four
parameters, all it seems is one linear stream of arguments. Any
structure the input may have had is lost.


And just what ARE A, C, and D?


It doesn't matter, and is not the concern of the shell. It should 
restrict itself to the basic parsing that may be necessary when 
parameters are separated by white-space and commas, if a parameter can 
contain white-space or commas. That usually involves surrounding the 
parameter with quotes.


One would be very annoyed if, reading a CSV file, where each of N values 
on a line correspond to a field of record, if one entry of "?LBC" 
expanded itself to a dozen entries, screwing everything up.


Shell command line processing shouldn't be attempting anything more than 
that.



If they are command options, on a Linux shell options appear as

X -A *.* -C -D

Even Windows command processor separates optional/command stuff via

X /A *.* /C /D

... or requires the parameter defining the file(s) to process as the last
arguments


Nonsense. If I pass that to a Python program that prints sys.argv, I get:

['a.py', '/A', '*.*', '/C', '/D']

Presumably a Windows programmer is grown up enough to make their own 
decisions as to what to do with that input.


All that's needed is one little standard library function to process 
sys.argc, with unexpanded parameters, into a list of expanded arguments, 
if that's really what someone wants (and then needing to trawl through 
it all looking for the options).


Then you get the best of both worlds.



X A C D *.*


So how do I do:

   gcc *.c -lm

The -lm needs to go at the end.

Presumably it now needs to check each parameter seeing if it resembles a 
file more than it does an option? And are options automatically taken 
care of, or is that something that, unlike the easier wildcards, need to 
be processed explicitly?


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[eryk sun]

On Mon, Dec 5, 2016 at 4:03 PM, Paul  Moore  wrote:
> 2. On Windows, the OS primitive takes a command line. The application is
> responsible for splitting it into arguments, if it wants to. Most do, for
> compatibility with the normal argv convention inherited via C from Unix. Many
> programs let the C runtime do that splitting for them - I don't recall if 
> Python
> does, or if it implements its own splitting these days.

Windows Python uses the CRTs parsed argument list. python.exe simply
uses the wmain entry point:

int
wmain(int argc, wchar_t **argv)
{
return Py_Main(argc, argv);
}

pythonw.exe uses the wWinMain entry point and the CRT's __argc and
__wargv variables:

int WINAPI wWinMain(
HINSTANCE hInstance,  /* handle to current instance */
HINSTANCE hPrevInstance,  /* handle to previous instance */
LPWSTR lpCmdLine, /* pointer to command line */
int nCmdShow  /* show state of window */
)
{
return Py_Main(__argc, __wargv);
}

python[w].exe doesn't link in wsetargv.obj, so it doesn't support
wildcard expansion.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 5:50 AM, BartC  wrote:
> So how do I do:
>
>gcc *.c -lm
>
> The -lm needs to go at the end.
>
> Presumably it now needs to check each parameter seeing if it resembles a
> file more than it does an option? And are options automatically taken care
> of, or is that something that, unlike the easier wildcards, need to be
> processed explicitly?

Actually, gcc processes a series of "things", where each thing could
be a file, a library definition, etc. So it recognizes "-lm" as a
library designation regardless of where it exists. It does NOT have to
be at the end specifically - you can have more file names after it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Lew Pitcher]

On Monday December 5 2016 11:23, in comp.lang.python, "BartC" 
wrote:

> On 05/12/2016 15:53, Chris Angelico wrote:
>> On Tue, Dec 6, 2016 at 2:41 AM, BartC  wrote:
>>>
>>> Are you saying that if someone executes:
>>>
>>>   subprocess.Popen(["python","a.py", "*"])
>>>
>>> the output will be: ['a.py','*']?
>>>
>>> In that case forget Windows vs. Linux, you now have a program that will
>>> get command parameters processed differently depending on whether it was
>>> invoked from a shell or not.
>>
>> Yes, that is correct. *SHELLS DO STUFF*. If you can't comprehend this,
>> you should get to know your shell. Try this:
>>
>> python a.py %PATH%
>>
>> subprocess.Popen(["python", "a.py", "%PATH%"])
>>
>> Would you expect those to do the same? If you do, prepare for Windows
>> to surprise you. If you don't, why do you keep expecting other shells
>> to do nothing?
> 
> You still don't get point. I write a program P, a native executable. It
> takes command line parameters but exactly what it gets depends on
> whether it's started from a 'shell' or from inside another program?

In Unix, it always has.


> I 
> don't want to worry about that stuff or exactly how it is invoked!

Then, I guess that you have a problem, don't you?

>  > subprocess.Popen(["python", "a.py", "%PATH%"])
> 
> Yes, %...% is an escape sequence. Those % signs are supposed to stand
> out and have been chosen not to clash with typical input.
> 
> And the end result of the transformation is, here, also a SINGLE thing;
> any parameter following will still be the second parameter, not the 14771th!
> 
> Are you saying that the * in ABC*EF also makes the whole thing some
> escape pattern?

If you ask the shell to parse the arguments, then, "YES, the * in ABC*EF makes
the argument a candidate for globbing".

> And one that could massively expand the number of parameters,

Yes

> pushing all the following ones out of the way, and making it 
> impossible to discover where these expanded parameters end and the
> normal ones recommence.

The best way to think about it is that all parameters are parameters, whether
derived from a glob input to a shell, or explicitly specified in the
invocation.

If you have a need for positional parameters, then either specify that your
code only accepts them in specific places, or find a way to disable globbing
(it can be done) and handle the expansion yourself, in your own code.

> If someone had thought this up now, it would rapidly be dismissed as
> being unworkable. But because it's been in Unix/Linux/whatever so long,
> no one can see anything wrong with it!

Primarily because there /IS/ nothing wrong with it.

-- 
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

On 12/05/2016 11:21 AM, Chris Angelico wrote:
> On Tue, Dec 6, 2016 at 5:02 AM, BartC  wrote:
>> If the syntax is:
>>
>>   program filespec
>>
>> or:
>>
>>   program filespec file
>>
>> how do you tell whether the last file in an argument list is the optional
>> 'file', or the last file of the expansion of 'filespec'?
> 
> Why should you care? I have used shell globbing to pass precisely two
> parameters to a program. More often, I use this syntax, which Windows
> simply doesn't support:
> 
> ffmpeg -i some-really-long-file-name.{avi,mkv}
> 
> to convert a file from one format to another. And if the .mkv file
> already exists, I can just use ".*" at the end, although that does
> depend on them being in alphabetical order. The shell is predictable
> and therefore useful. This trick is guaranteed to work, no matter what
> program I'm using. On Windows, I have to hope that the program expands
> these notations correctly.

Agreed. I do this sort of trick all the time, even when I want to pass
just a single file to a program.  I often use expansion for paths as well:

somecommand /path/to/somelongname*withendpart/myepisode*2x03*mp4

I could use tab expansion as I go, but tab expansion might reveal
several options to pick from, requiring a few additional keystrokes to
arrive at the path I want.  Globs save that typing.  And shells are
smart enough to expand the expression even after several wildcards have
been used.  It's a win win.  And the program doesn't have to know
anything about it to work.

Now I usually can use the same expressions in cmd.exe. But I find other
parts of Windows' command-line parsing to really strange, particularly
when it comes to spaces in the filenames.  I'm not sure if this is
cmd.exe's fault or just the win32 api.

In the Unix world, there are times when you don't want shell expansion.
For example, when dealing with ssh or rsync, you most often want the
wildcards and other expression characters to be passed through to the
remote process.  By default zsh makes you escape them all so they won't
attempt expansion locally.  If zsh can't expand an expression locally,
it gives you an error.  This can be controlled with a config option.

Bash on the other hand, passes through any expressions it can't expand
as is.  So under certain circumstances, "python a*.py" would indeed pass
"a*.py" to the program.  But 99% of the time this does not matter.
Whether the glob expands to zero files or the program attempts to open
"a*.py" as a literal file, the effect is usually the same.  Most unix
shells default to this behavior, which is useful when using rsync.

I'd far rather leave it up to the shell to do expansion than to do it
myself in a program. It allows consistency within the shell experience.
If people want the zsh idea of erring out on an expansion, they can have
that.  Or they can use a shell that behaves like bash.  Either way there
is a consistency there that's just not there on Windows.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 18:34, eryk sun wrote:

On Mon, Dec 5, 2016 at 3:41 PM, BartC  wrote:


Are you saying that if someone executes:

  subprocess.Popen(["python","a.py", "*"])

the output will be: ['a.py','*']?

In that case forget Windows vs. Linux, you now have a program that will get
command parameters processed differently depending on whether it was invoked
from a shell or not.

Or a program that sometimes will see "*" as an argument, and sometimes a big
list of files that merges into all the other arguments.


If it sees "*", it will try to open a file named "*".


And people still say that the way Windows works is crazy!

 That's a valid

filename in Unix, but it should be avoided.


No, it should be prohibited, if the file API and/or shell assign special 
significance to it.



I don't understand your concern regarding Unix shell scripting syntax.
On Windows, are you as troubled that environment variables such as
%USERNAME% get expanded by cmd.exe, but not by CreateProcess?


No, because I never use %USERNAME%, and it is obviously something 
associated with Windows' Batch language which I hardly ever use.


Not like file wildcards, which I use all the time, and don't associate 
them exclusively with a shell.


I also never expect any parameter that contains ? and * to be expanded 
then and there to whatever list of files there might happen to be. Maybe 
the files haven't been created yet, maybe the parameter has got NOTHING 
TO DO with filenames.


(For example, in Windows:

   >ren *.c *.d

Rename all .c files to .d files. None of the .d files exist (or, because 
of the point below, some isolated .d file exists). I wouldn't be able to 
emulate such a command under Linux, not without writing:


  rename *.c "*.d"

or even:

  rename "*.c" "*.d"

since, if the user forgets the second parameter, and there were two .c 
files, it would think I wanted to rename one .c file to another.)


But that leads to another weird behaviour I've just observed: if a 
parameter contains a filespec with wildcards, and there is at least one 
matching file, then you get a list of those files.


However, if there are no matching files, you get the filespec unchanged.

Now, what is a program supposed to with that? It seems it has to deal 
with wildcards after all. But also some behaviours can become erratic. 
This program asks for the name of a DAB radio station:


  >station ?LBC

Fine, the input is ?LBC (which is an actual station name where I live). 
But then at some point a file ALBC comes into existence; no connection. 
Now the same line gives the input ALBC, to perplex the user!



Every command-line shell that I've ever used is also a quirky
scripting language.


It seems shell language authors have nothing better to do than adding 
extra quirky features that sooner or later are going to bite somebody 
on the arse. Mainly I need a shell to help launch a program and give it 
some basic input; that's all.


(The people who think up obscure hot-key combinations belong in the same 
camp. I keep hitting key combinations that do weird things, then have to 
find a way out. One combination turned the display upside-down! Then I 
had to turn the monitor upside down while I investigated solutions.)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Printing a generator returns "", need to print its values

[Irmen de Jong]

On 5-12-2016 19:39, vmaha...@centerpointmedia.com wrote:
> On Wednesday, November 16, 2016 at 3:25:39 PM UTC-5, Peter Otten wrote:
>> vmaha...@centerpointmedia.com wrote:
>>
>>> I am running Python2.7, wherein I am running the following price of code:
>>>
>>> y = m.predict(input_fn=lambda:input_fn(df_predict), as_iterable=True)
>>> print ('Predictions: {}'.format(str(y)))
>>>
>>> The output of the following is "">> 0x7f0476373e10>"
>>>
>>> However, the desired output must be in the format [1 0 0 1 1 0 0 1].
>>>
>>> Can someone help me print the output in this format
>>
>> You give no context, but my suggestion would be to try and omit the 
>>
>> as_iterable=True
>>
>> part...
> 
> 
> Can someone help me print a generator object?
> 

Have you tried Peter's suggestion?

If that didn't work (but what is that as_iterable argument for then?) just 
iterate over
the result and print each value as it comes out of the generator:

print("Predictions:")
for value in y:
print(value)


Irmen

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 6:08 AM, Michael Torrie  wrote:
> Agreed. I do this sort of trick all the time, even when I want to pass
> just a single file to a program.  I often use expansion for paths as well:
>
> somecommand /path/to/somelongname*withendpart/myepisode*2x03*mp4
>

"somecommand" is "vlc", isn't it :D

And I agree. I do this too. If you tab complete it, you can't as
easily use the up arrow. Compare:

vlc /video/Music/ASongToSingO/203*
vlc /video/Music/ASongToSingO/203\ Bugs\ Bunny\ at\ the\ Gaiety.ogg

The latter came from "203". Now, you want to play track 216
"Trial by Ruddy Patience" next. With the first one, it's up, left,
backspace two, patch in the new number, hit enter. With the second,
you have to backspace the whole title, then put in 216, and
tab-complete it again. Now, VLC on Windows is probably capable of
expanding globs, but why should it bother? Bash gives the exact same
result for both of those commands.

Plus, the Windows method depends on the globbing characters being
INVALID in file names. You absolutely cannot create a file called
"What's up, doc?.mkv" on Windows. And I'm not sure what other
characters are valid - are you allowed quotes? If so, how do you quote
the quote character? And what happens with network mounts?

With the Unix model, where the shell does the work, it's the shell's
job to provide an escaping mechanism. The shell doesn't have to be
externally consistent (although it helps a lot), but it can be
perfectly internally consistent. The *system* simply passes everything
along as-is, as an array of arguments.

Windows gives you less freedom than Unix does.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

On 12/05/2016 11:50 AM, BartC wrote:
> It doesn't matter, and is not the concern of the shell. It should 
> restrict itself to the basic parsing that may be necessary when 
> parameters are separated by white-space and commas, if a parameter can 
> contain white-space or commas. That usually involves surrounding the 
> parameter with quotes.

That is just a difference of opinion and what you're used to.  A shell's
primary concern on my computer is providing a great environment for me
to work in, and for that, shell expansion is very important and useful
to me.  Also, in the Unix world (can't speak for Windows), we assemble
lots of little bash scripts into individual executable units, and string
them together. In other words, many "commands" we use in Unix are
actually little scripts themselves.  So having each and every little
script have to do its own glob expansion would be very tedious and
redundant. Moving that up to the shell makes an incredible amount of
sense given this structure.

> One would be very annoyed if, reading a CSV file, where each of N values 
> on a line correspond to a field of record, if one entry of "?LBC" 
> expanded itself to a dozen entries, screwing everything up.

I have no idea why you even put this in here.  What does reading a CSV
file have to do with glob expansion?  Sorry Bart, but you're grasping at
straws with that one.

> Shell command line processing shouldn't be attempting anything more than 
> that.

In your opinion.  Clearly there's a whole world out there that thinks
differently, and has their own good reasons.

> 
>>  If they are command options, on a Linux shell options appear as
>>
>> X -A *.* -C -D
>>
>> Even Windows command processor separates optional/command stuff via
>>
>> X /A *.* /C /D
>>
>> ... or requires the parameter defining the file(s) to process as the last
>> arguments
> 
> Nonsense. If I pass that to a Python program that prints sys.argv, I get:
> 
> ['a.py', '/A', '*.*', '/C', '/D']
> 
> Presumably a Windows programmer is grown up enough to make their own 
> decisions as to what to do with that input.
> 
> All that's needed is one little standard library function to process 
> sys.argc, with unexpanded parameters, into a list of expanded arguments, 
> if that's really what someone wants (and then needing to trawl through 
> it all looking for the options).

Sure I understand that, and I can understand the appeal.  I just
disagree that it's this big issue you seem to think it is.

> Then you get the best of both worlds.
> 
> 
>> X A C D *.*
> 
> So how do I do:
> 
> gcc *.c -lm
> 
> The -lm needs to go at the end.
> 
> Presumably it now needs to check each parameter seeing if it resembles a 
> file more than it does an option? And are options automatically taken 
> care of, or is that something that, unlike the easier wildcards, need to 
> be processed explicitly?

To add to what Chris said, all the different parameters have to be
looked at anyway, so I don't understand why it's such a burden to scan
them one by one and if they are a filename, add them to a list, if they
are a command-line argument, do something different.  And in the case of
gcc, one or more globs could appear, in any order.  So in Windows you'd
have to look at each argument anyway, and do n number of manual
expansions for non-argument parameters.  So this particular argument of
yours is rather weak.

Not having glob expansion be in individual programs makes things much
more explicit, from the program's point of view.  It provides an
interface that takes filename(s) and you provide them, either explicitly
(via popen with no shell), or you can do it implicitly but in an
interactive way via the shell using expansion.  Personal preference but
I believe it's a much better way because it's explicit from the
program's point of view as there's no question about the program's behavior.

Different strokes for different folks as they say.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Chris Angelico]

On Tue, Dec 6, 2016 at 6:24 AM, BartC  wrote:
>> If it sees "*", it will try to open a file named "*".
>
>
> And people still say that the way Windows works is crazy!
>
>  That's a valid
>>
>> filename in Unix, but it should be avoided.
>
>
> No, it should be prohibited, if the file API and/or shell assign special
> significance to it.
>

That implies that the file system, file API, and shell are all bound
together. You have to forbid in file names *any* character that has
*any* significance to *any* shell. That means no quotes, no asterisks
or question marks, no square brackets, no braces, etc, etc, etc.
or, alternatively, restrict your shells to all perform the exact same
expansion - or no expansion at all, and then restrict all your
_applications_ to perform the same expansion.

Good luck.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

Bored to day since it's -20 and I don't want to work outside.

On 12/05/2016 12:24 PM, BartC wrote:
>> If it sees "*", it will try to open a file named "*".
> 
> And people still say that the way Windows works is crazy!
> 
>   That's a valid
>> filename in Unix, but it should be avoided.
> 
> No, it should be prohibited, if the file API and/or shell assign special 
> significance to it.

Why? What does it matter to the file system what characters are in the
filename?

> 
>> I don't understand your concern regarding Unix shell scripting syntax.
>> On Windows, are you as troubled that environment variables such as
>> %USERNAME% get expanded by cmd.exe, but not by CreateProcess?
> 
> No, because I never use %USERNAME%, and it is obviously something 
> associated with Windows' Batch language which I hardly ever use.

cmd.exe *IS* the the "Windows' Batch" language.  So if you use cmd.exe,
you use this "Batch" language all the time.

> Not like file wildcards, which I use all the time, and don't associate 
> them exclusively with a shell.
>
> 
> I also never expect any parameter that contains ? and * to be expanded 
> then and there to whatever list of files there might happen to be. Maybe 
> the files haven't been created yet, maybe the parameter has got NOTHING 
> TO DO with filenames.
> 
> (For example, in Windows:
> 
> >ren *.c *.d
> 
> Rename all .c files to .d files. None of the .d files exist (or, because 
> of the point below, some isolated .d file exists). I wouldn't be able to 
> emulate such a command under Linux, not without writing:
> 
>rename *.c "*.d"
> 
> or even:
> 
>rename "*.c" "*.d"

Wow. Does that actually work?  And work consistently?  How would it
handle globs like this:

renamae a*b??c*.c *.d

I can't think of any way to do that in a consistent way.  I can see it
working for the simple cases. Must have quite a bit of logic in rename
to handle all the corner cases.

Such a syntax, even with quoting, would be highly unusual to see in the
Unix world.  I think I'd rather just use a simple loop and have the
rename be explicit so I know it's working (and yes I do this all the
time) as I expect it to.

> since, if the user forgets the second parameter, and there were two .c 
> files, it would think I wanted to rename one .c file to another.)
> 
> But that leads to another weird behaviour I've just observed: if a 
> parameter contains a filespec with wildcards, and there is at least one 
> matching file, then you get a list of those files.
> 
> However, if there are no matching files, you get the filespec unchanged.

That's true for many shells (but not zsh by default which errors out).

> Now, what is a program supposed to with that? It seems it has to deal 
> with wildcards after all. But also some behaviours can become erratic. 
> This program asks for the name of a DAB radio station:

Again, it doesn't really matter.  If the program was looking for
filenames, the program will try to open the file as given ("a*.txt") and
if it succeeds it succeeds, if it fails it fails.  The program need not
care. Why should it?

> 
>>station ?LBC
> 
> Fine, the input is ?LBC (which is an actual station name where I live). 
> But then at some point a file ALBC comes into existence; no connection. 
> Now the same line gives the input ALBC, to perplex the user!

Why would a file come into existence? I thought you said it refers to a
station identifier. No wonder you have such confused users if you're
creating files when you should be opening a URL or something.

I would expect in this case that the "station" program would return an
error, something like:

Error! Cannot find station "ALBC"

If station ids were supposed to start with a wildcard character, I'd
probably have to make a note in the help file that the station
identifiers have to be escaped or placed in quotes.  Or change my logic
to not require this ? character (if it was part of all identifiers we
can drop it safely).

> 
>> Every command-line shell that I've ever used is also a quirky
>> scripting language.
> 
> It seems shell language authors have nothing better to do than adding 
> extra quirky features that sooner or later are going to bite somebody 
> on the arse. Mainly I need a shell to help launch a program and give it 
> some basic input; that's all.

I think you'd quickly find that such a shell would be very non-useful.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[John Gordon]

In  DFS  writes:

> > Shells don't just repeat the characters you type, they interpret them.

> Yes, I see that now.  I still don't think bash/shell should alter the 
> input going to another program.

But that's one of the reasons *to* use a shell!

  ls *.cERROR No file named "*.c"
  ls > output   ERROR No file named ">"
  rm 'a file name'  ERROR No file named "'a file name'"
  cd ~/foo/bar  ERROR No file named "~/foo/bar"
  cat $HOME/file.txtERROR No file named '$HOME/file.txt' 
  vi $(grep -l foo *.txt)   ERROR No file named '$(grep -l foo *.txt)'

None of these commands would work if bash didn't "alter the input going to
another program".

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Random832]

On Mon, Dec 5, 2016, at 14:48, Michael Torrie wrote:
> Wow. Does that actually work?  And work consistently?  How would it
> handle globs like this:

The rules are simpler than you're probably thinking of. There's actually
no relationship between globs on the left and on the right. Globs on the
left simply select the files to rename as normal, the glob pattern
doesn't inform the renaming operation.

A question mark on the right takes the character from *that character
position* in the original filename. That is, if you have "abc", "rename
ab? ?de" renames it to "ade", not "cde".

A star on the right takes all remaining characters. If you include a
dot, the "name" (everything before the last dot) and "extension" of the
file are considered separate components [so in adddition to rename *.c
*.d, renaming a.* b.* where you have a.py, a.pyc, a.pyo will work].

But if you don't include a dot, for example "rename abcde.fgh xyz*", it
will rename to xyzde.fgh .
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Marko Rauhamaa]

Chris Angelico :

> On Tue, Dec 6, 2016 at 4:23 AM, Marko Rauhamaa  wrote:
>> Bash is nice, too nice. It makes it easy to write code that's riddled
>> with security holes. The glorious Unix tradition is to ignore the
>> pitfalls and forge ahead come what may.
>
> Bash assumes that the person typing commands has the full power to
> execute commands. I'm not sure what you mean by "security holes",
> unless it's passing text through bash that came from people who aren't
> allowed to type commands.

Let's mention a few traps:

 * Classic shell programming is built on the concept of string lists. A
   list of three file names could be stored in a variable as follows:

  files="a.txt b.txt c.txt"

   or maybe:

  files=$*

   Trouble is, whitespace is not a safe separator because it is a valid
   character in filenames.

   The only safe way out is to use bash's arrays, which are a deviation
   of the classic foundation. That's why most (?) shell scripts just
   ignore the possibility of whitespace in filenames.

 * A common, classic pattern to pipeline data is something like this:

  find_files |
  while read filename; do
  ... do a thing or two with "$filename" ...
  done

   Unfortunately, the pattern suffers from several problems:

1. A filename can legally contain a newline.

2. A filename can legally begin and end with whitespace which is
   stripped by the "read" builtin.

3. The "read" builtin treats the backslash as an escape character
   unless the "-r" option is specified. The backslash is a valid
   character in a filename.

 * The classic "test" builtin (or "[ ... ]") has ambigous syntax:

  if [ -e "$filename" -o -e "$filename.bak" ]; then
  ...
  fi

   results in a syntax error if "$filename" should be "=".

 * This fails gracefully:

  local x
  x=$(false) || exit

   This doesn't fail at all:

  local x=$(false) || exit


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

On 12/05/2016 01:09 PM, Random832 wrote:
> The rules are simpler than you're probably thinking of. There's actually
> no relationship between globs on the left and on the right. Globs on the
> left simply select the files to rename as normal, the glob pattern
> doesn't inform the renaming operation.
> 
> A question mark on the right takes the character from *that character
> position* in the original filename. That is, if you have "abc", "rename
> ab? ?de" renames it to "ade", not "cde".
> 
> A star on the right takes all remaining characters. If you include a
> dot, the "name" (everything before the last dot) and "extension" of the
> file are considered separate components [so in adddition to rename *.c
> *.d, renaming a.* b.* where you have a.py, a.pyc, a.pyo will work].
> 
> But if you don't include a dot, for example "rename abcde.fgh xyz*", it
> will rename to xyzde.fgh .

Ahh.  Good explanation.  Thank you.  So in the case of rename, the
second argument isn't actually a glob at all (even though it looks like
one).  A useful simplification to be sure, though not one I'd want to
emulate (it relies on Windows' idea of file extensions to really work
correctly).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 19:48, Michael Torrie wrote:

Bored to day since it's -20 and I don't want to work outside.

On 12/05/2016 12:24 PM, BartC wrote:



(For example, in Windows:

>ren *.c *.d

Rename all .c files to .d files. None of the .d files exist (or, because
of the point below, some isolated .d file exists). I wouldn't be able to
emulate such a command under Linux, not without writing:

   rename *.c "*.d"

or even:

   rename "*.c" "*.d"


Wow. Does that actually work?  And work consistently?  How would it
handle globs like this:

renamae a*b??c*.c *.d


That's not really the point, which is that a parameter may use * and ? 
which, even if they are related to file names, may not refer to actual 
input files in this directory. Here, they are output files. But this is 
just one of dozens of reasons why automatic expansion of such patterns 
is not desirable.


(And yes sometimes it works, for simple cases, and sometimes it screws 
up. But it is up to the program how it defines its input and what it 
will do with it. It shouldn't be up to the shell to expand such names in 
ALL circumstances whether that is meaningful to the app or not.)



But that leads to another weird behaviour I've just observed: if a
parameter contains a filespec with wildcards, and there is at least one
matching file, then you get a list of those files.

However, if there are no matching files, you get the filespec unchanged.


That's true for many shells (but not zsh by default which errors out).


Now, what is a program supposed to with that? It seems it has to deal
with wildcards after all. But also some behaviours can become erratic.
This program asks for the name of a DAB radio station:


Again, it doesn't really matter.  If the program was looking for
filenames, the program will try to open the file as given ("a*.txt") and
if it succeeds it succeeds, if it fails it fails.  The program need not
care. Why should it?


Really, the program should just fail? It seems to me that the program is 
then obliged to deal with wildcards after all, even if just to detect 
that wildcards have been used and that is reason for the error. But my 
example was how that behaviour can stop programs working in random ways.



Fine, the input is ?LBC (which is an actual station name where I live).
But then at some point a file ALBC comes into existence; no connection.
Now the same line gives the input ALBC, to perplex the user!


Why would a file come into existence? I thought you said it refers to a
station identifier. No wonder you have such confused users if you're
creating files when you should be opening a URL or something.


The file is irrelevant. It's just a bit of junk, or something that is 
used for some other, unrelated purpose.


The important thing is that creation of that file FOR ANY REASON could 
screw up a program which is asking for input that is unrelated to any files.



I would expect in this case that the "station" program would return an
error, something like:

Error! Cannot find station "ALBC"


And that's the error! But how is the user supposed to get round it? It's 
an error due entirely to unconditional shell expansion of any parameter 
that looks like it might be a filename with wildcards.



If station ids were supposed to start with a wildcard character, I'd
probably have to make a note in the help file that the station
identifiers have to be escaped or placed in quotes.


I've no idea why that station starts with "?" (it appears on the station 
list, but have never tried to listen to it). But this is just an example 
of input that might use ? or * that has nothing to do files, yet can be 
affected by the existence of some arbitrary file.


And somebody said doing it this way is more secure than how Windows does 
things!



It seems shell language authors have nothing better to do than adding
extra quirky features that sooner or later are going to bite somebody
on the arse. Mainly I need a shell to help launch a program and give it
some basic input; that's all.


I think you'd quickly find that such a shell would be very non-useful.


For about five minutes, until someone produces an add-on with the 
missing features.


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Detect Linux Runlevel

[Wildman via Python-list]

I there a way to detect what the Linux runlevel is from
within a Python program?  I would like to be able to do
it without the use of an external program such as 'who'
or 'runlevel'.

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[BartC]

On 05/12/2016 19:29, Michael Torrie wrote:

On 12/05/2016 11:50 AM, BartC wrote:



So how do I do:

gcc *.c -lm

The -lm needs to go at the end.

Presumably it now needs to check each parameter seeing if it resembles a
file more than it does an option? And are options automatically taken
care of, or is that something that, unlike the easier wildcards, need to
be processed explicitly?


This was a response to someone saying the wildcard param needs to be at 
the end. There need be no such restriction if handled properly (ie. no 
auto-expansion).


But a similar example, suppose a syntax is:

  appl *.* [options]

but one of the files in the list is called "-lm", or some other option 
that can do things the user didn't want (with gcc, -lm is harmless).


Without expansion, input is easy to parse: filespec, followed by 
optional options. But with expansion, now you have to decide if a 
particular argument is an option, or a filename.


And if there's an error in an option, you may have to abort, which means 
throwing away that list of files which, in some cases, can run into 
millions.



Not having glob expansion be in individual programs makes things much
more explicit, from the program's point of view.  It provides an
interface that takes filename(s) and you provide them, either explicitly
(via popen with no shell), or you can do it implicitly but in an
interactive way via the shell using expansion.  Personal preference but
I believe it's a much better way because it's explicit from the
program's point of view as there's no question about the program's behavior.

Different strokes for different folks as they say.


I must have given ten or twenty scenarios where such auto-expansion is 
problematical. And yet people are still in denial. It's been in Unix for 
decades therefore there's nothing wrong with it!


But maybe people simply avoid all the situations that cause problems. 
Interfaces are specified in a certain manner, given that input can be 
any long stream of filenames and/or options with no order and no 
positional significance. Non-file parameters that use ? or * are 
prohibited. You can't do the equivalent of:


 >DIR *.b *.c

And get a list of *.b files, with a heading and summary lines, then a 
list of *.c files with its own heading and summary. It would be one 
monolithic list.


So everyone is working around the restrictions and the problems. Which 
is exactly what I would have to do.


That doesn't change the fact that the Windows approach is considerably 
more flexible and allowing more possibilities.


--
Bartc


--
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 16:25:56 -0500, DFS wrote:

> On 12/05/2016 03:58 PM, Wildman wrote:
>> I there a way to detect what the Linux runlevel is from
>> within a Python program?  I would like to be able to do
>> it without the use of an external program such as 'who'
>> or 'runlevel'.
> 
> 
> Why not?
> 
> '>>> import os
> '>>> os.system("systemctl get-default")
> graphical.target
> 
> 
> 
> systemd 'graphical.target' corresponds to the old runlevel 5.

Thanks but I knew about systemctl.  As I already said my
goal is to do it without the use of an external program.

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Lew Pitcher]

On Monday December 5 2016 16:25, in comp.lang.python, "DFS" 
wrote:

> On 12/05/2016 03:58 PM, Wildman wrote:
>> I there a way to detect what the Linux runlevel is from
>> within a Python program?  I would like to be able to do
>> it without the use of an external program such as 'who'
>> or 'runlevel'.
> 
> 
> Why not?
> 
> '>>> import os
> '>>> os.system("systemctl get-default")
> graphical.target

Because

  $ cat rlevel.py
  import os
  os.system("systemctl get-default")
  16:36 $ python rlevel.py
  sh: systemctl: command not found
  16:36 $ 


> systemd 'graphical.target' corresponds to the old runlevel 5.

Yes? So?

The OP asked for the runlevel, not the systemd target.


-- 
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Marko Rauhamaa]

Wildman :
> Thanks but I knew about systemctl. As I already said my goal is to do
> it without the use of an external program.

Inspect:

   https://github.com/systemd/systemd/blob/master/src/systemctl/systemctl.c>

In particular:

   get_state_one_unit()

Then, proceed to:

   https://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html>


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Chris Angelico]

On Tue, Dec 6, 2016 at 8:38 AM, Lew Pitcher
 wrote:
> The OP asked for the runlevel, not the systemd target.

Runlevels don't exist in systemd. And systemd targets don't exist in
Upstart. The question "what runlevel are we in" does not make sense
unless you're using an init system that works on the basis of
runlevels (eg sysvinit).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Tim Chase]

On 2016-12-05 14:58, Wildman via Python-list wrote:
> I there a way to detect what the Linux runlevel is from
> within a Python program?  I would like to be able to do
> it without the use of an external program such as 'who'
> or 'runlevel'.

You can use something like

https://gist.github.com/likexian/f9da722585036d372dca

to parse the /var/run/utmp contents.  Based on some source-code
scrounging, it looks like you want the first field to be "1" for the
"runlevel" account.  To extract the actual runlevel, you can take
the PID value from the second column ("53" in my example here) and
take it's integer value mod 256 (AKA "& 0xff") to get the character
value.  So chr(int("53") & 0xff) returns "5" in my case, which is my
runlevel.

Additional links I found helpful while searching:

https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
https://github.com/garabik/python-utmp

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Marko Rauhamaa]

Chris Angelico :

> On Tue, Dec 6, 2016 at 8:38 AM, Lew Pitcher
>  wrote:
>> The OP asked for the runlevel, not the systemd target.
>
> Runlevels don't exist in systemd. And systemd targets don't exist in
> Upstart. The question "what runlevel are we in" does not make sense
> unless you're using an init system that works on the basis of
> runlevels (eg sysvinit).

In fact, systemd is not an init system for Linux. Linux is the kernel of
the systemd operating system. Systemd is the

  One Ring to rule them all, One Ring to find them,
   One Ring to bring them all and in the darkness bind them


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 23:59:48 +0200, Marko Rauhamaa wrote:

> Wildman :
>> Thanks but I knew about systemctl. As I already said my goal is to do
>> it without the use of an external program.
> 
> Inspect:
> 
>https://github.com/systemd/systemd/blob/master/src/systemctl/systemctl.c>
> 
> In particular:
> 
>get_state_one_unit()

Too bad I don't speak C.  I am an amateur programmer and most or all
my experience has been with assembly and various flavors of BASIC,
including VB and PowerBASIC.  I did look over the code but I guess
I'm just a rebel without a clue. 

> Then, proceed to:
> 
>https://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html>
> 
> 
> Marko

Even if I could ‎figure this out, I can't depend on systemd being
installed.  The way things are going that may change tho.  I have
bookmarked the pages for possible future use.

Thank you, I do appreciate your post.

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

On 12/05/2016 01:35 PM, BartC wrote:
>>> It seems shell language authors have nothing better to do than adding
>>> extra quirky features that sooner or later are going to bite somebody
>>> on the arse. Mainly I need a shell to help launch a program and give it
>>> some basic input; that's all.
>>
>> I think you'd quickly find that such a shell would be very non-useful.
> 
> For about five minutes, until someone produces an add-on with the 
> missing features.

Which will re-implement everything you stripped out. :)

As I've gotten older I've learned the truth of this quotation:
"Those who do not understand UNIX are condemned to reinvent it, poorly."
-- Henry Spencer


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Michael Torrie]

On 12/05/2016 03:34 PM, Wildman via Python-list wrote:
> Too bad I don't speak C.  I am an amateur programmer and most or all
> my experience has been with assembly and various flavors of BASIC,
> including VB and PowerBASIC.  I did look over the code but I guess
> I'm just a rebel without a clue. 

May I ask what you are trying to accomplish?  Why does the runlevel matter?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Michael Torrie]

On 12/05/2016 03:29 PM, Marko Rauhamaa wrote:
> Chris Angelico :
> 
>> On Tue, Dec 6, 2016 at 8:38 AM, Lew Pitcher
>>  wrote:
>>> The OP asked for the runlevel, not the systemd target.
>>
>> Runlevels don't exist in systemd. And systemd targets don't exist in
>> Upstart. The question "what runlevel are we in" does not make sense
>> unless you're using an init system that works on the basis of
>> runlevels (eg sysvinit).
> 
> In fact, systemd is not an init system for Linux. Linux is the kernel of
> the systemd operating system. Systemd is the
> 
>   One Ring to rule them all, One Ring to find them,
>One Ring to bring them all and in the darkness bind them

Well I for one am glad of the systemd init system inside of my Linux
operating system.  I would far rather deal with service ini files than
long arcane bash scripts that often re-implement (poorly) things like
pid files, and attempts to prevent more than one instance from running.
You were the one that posted earlier today about the many perils of
programming complicated scripts in bash. Init scripts pretty much hit on
all of those gotchas!  It boggles my mind that people would actually
argue in favor of maintaining such things (and yes I have done it on
many servers over the years and it was always a pain to deal with custom
daemons).

I don't use very many of the systemd modules (most are not even
installed on my machine), nor will I ever need them. In fact I still
have rsyslog running.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Lew Pitcher]

On Monday December 5 2016 17:24, in comp.lang.python, "Chris Angelico"
 wrote:

> On Tue, Dec 6, 2016 at 8:38 AM, Lew Pitcher
>  wrote:
>> The OP asked for the runlevel, not the systemd target.
> 
> Runlevels don't exist in systemd. And systemd targets don't exist in
> Upstart. The question "what runlevel are we in" does not make sense
> unless you're using an init system that works on the basis of
> runlevels (eg sysvinit).

I repeat: The OP asked for the runlevel, not the systemd target.

That should tell you that an answer involving systemd "does not make sense".

To the OP: as others have said, the file
  /var/run/utmp
contains various records, including the RUN_LVL (runlevel) record. You can
find some documentation in utmp(3), including a record layout, and an values
list.

I don't know that python includes a standard or builtin method to parse the
utmp file; to retrieve the runlevel, you may have to code your own access
routine as part of your python code, or resort to invoking an external
program like who(1).

HTH
-- 
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Printing a generator returns "", need to print its values

[Steve D'Aprano]

On Tue, 6 Dec 2016 05:39 am, vmaha...@centerpointmedia.com wrote:

> Can someone help me print a generator object?

The same way as you print any other object:

print(obj)  # Python 3

print obj  # Python 2


But what you're actually asking for is a way to print the values produced by
the generator object. You do that the same way as you would print the
values produced by any sequence or iterable. You can print one value per
line:

for value in obj:
print(value)


Or you can convert to a list, and then print the list:

print(list(obj))


Or you can format it yourself, any way you like, by writing some code:


display_string = '::'.join(str(value).upper() for value in obj)
print('[[' + display_string + ']]')




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[eryk sun]

On Mon, Dec 5, 2016 at 4:49 PM, Steve D'Aprano
 wrote:
>
> You've never used cmd.com or command.exe? "The DOS prompt"?

The default Windows shell is "cmd.exe", and it's informally called the
"Command Prompt", not "DOS Prompt". In Windows 9x it was accurate to
say DOS prompt, since the shell was COMMAND.COM, which used DOS system
calls. But that branch of Windows has been dead for over a decade.

> Even the DOS prompt supports some level of globbing. Its been a while since
> I've used the DOS prompt in anger, but I seem to recall being able to do
> things like:
>
> dir a*

"dir" is a built-in command. It calls FindFirstFileExW to list each
directory that it walks over.

FindFirstFileExW converts the glob to a form supported by the system
call NtQueryDirectoryFile. In this case that's simply "a*". In other
cases it tries to match the behavior of MS-DOS globbing, which
requires rewriting the pattern to use DOS_STAR ('<'), DOS_QM ('>'),
and DOS_DOT ('"'). To simplify the implementation, the five wildcard
characters are reserved. They're not allowed in filenames.

The I/O manager leaves the implementation up to the filesystem driver.
NtQueryDirectoryFile gets dispatched to the driver as an I/O request
packet with major function IRP_MJ_DIRECTORY_CONTROL and minor function
IRP_MN_QUERY_DIRECTORY. The driver does most of the work, including
filtering the directory listing by the FileName argument. But a driver
writer doesn't have to reinvent the wheel here; the filesystem runtime
library has string comparison functions that support wildcards, such
as FsRtlIsNameInExpression.

> *every single command and application* has to re-implement its own globbing,
> very possibly inconsistently.

C/C++ programs can link with wsetargv.obj to support command-line globbing.

For Python, this dependency can be added in PCBuild/python.vcxproj in
the linker configuration:


  Console
  200
  0x1d00
  wsetargv.obj


For example:

C:\Temp\test>dir /b
a.txt
b.dat
c.bin

C:\Temp\test>python -c "import sys;print(sys.argv)" *.txt *.dat
['-c', 'a.txt', 'b.dat']
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 15:39:24 -0700, Michael Torrie wrote:

> On 12/05/2016 03:34 PM, Wildman via Python-list wrote:
>> Too bad I don't speak C.  I am an amateur programmer and most or all
>> my experience has been with assembly and various flavors of BASIC,
>> including VB and PowerBASIC.  I did look over the code but I guess
>> I'm just a rebel without a clue. 
> 
> May I ask what you are trying to accomplish?  Why does the runlevel matter?

Of course you may.  It is part of a program I'm working on
that reports different information about the Linux system it
is running on.  A program similar to inxi.  And I am trying
to write it without using external programs, where possible.

I am a hobby programmer and I've been trying to learn python
for a few months now.  The program is 'worthlessware' but it
is a 'learning experience' for me.  A friend wrote a similar
program in Bash script and I have been working on translating
it to Python.

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Marko Rauhamaa]

Michael Torrie :

> On 12/05/2016 03:29 PM, Marko Rauhamaa wrote:
>> In fact, systemd is not an init system for Linux. Linux is the kernel of
>> the systemd operating system. Systemd is the
>> 
>>   One Ring to rule them all, One Ring to find them,
>>One Ring to bring them all and in the darkness bind them
>
> Well I for one am glad of the systemd init system inside of my Linux
> operating system. I would far rather deal with service ini files than
> long arcane bash scripts that often re-implement (poorly) things like
> pid files, and attempts to prevent more than one instance from
> running.

The situation before systemd *was* atrocious. Most of the protests
against systemd were misplaced. They advocated the ancient hacker
culture that placed the supreme authority on the System Administrator,
who fashioned the system into his own image.

I appreciate that finally there was a bold soul who took the point of
view of an Architect and laid down some higher-level principles for the
whole system to abide by.

Unfortunately, I am not wholly impressed by the end result. Mogadishu
has been replaced by Pyongyang. Some age-old Unix principles have been
abandoned without clear justification. For example, I was appalled to
find out that a systemd unit can be configured to react on the exit
status of a child process of a daemon.

Also, now D-Bus is a fixture for every daemon writer.

> You were the one that posted earlier today about the many perils of
> programming complicated scripts in bash. Init scripts pretty much hit on
> all of those gotchas!

Guess what -- the unit files come with their own gotchas. For example,
there is no simple way to convert an arbitrary pathname into an
ExecStart entry of a unit file.

The .ini file format was a lousy choice. Why not choose JSON in this day
and age?

Yes, the SysV init system had become a jungle. Still, I wish it had been
replaced with a rigorous protocol. The units should have been programs
that comply with given contracts.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Marko Rauhamaa]

Michael Torrie :
> As I've gotten older I've learned the truth of this quotation:
> "Those who do not understand UNIX are condemned to reinvent it, poorly."
> -- Henry Spencer

I thought you kinda liked systemd...


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

On 12/05/2016 04:38 PM, Marko Rauhamaa wrote:
> Michael Torrie :
>> As I've gotten older I've learned the truth of this quotation:
>> "Those who do not understand UNIX are condemned to reinvent it, poorly."
>> -- Henry Spencer
> 
> I thought you kinda liked systemd...

Yup I do.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Michael Torrie]

On 12/05/2016 04:37 PM, Marko Rauhamaa wrote:
> Unfortunately, I am not wholly impressed by the end result. Mogadishu
> has been replaced by Pyongyang. Some age-old Unix principles have been
> abandoned without clear justification. For example, I was appalled to
> find out that a systemd unit can be configured to react on the exit
> status of a child process of a daemon.

I have yet to see any evidence of this Pyonguang situation.

What is wrong with reacting to the exit status of a daemon's child process?

> Also, now D-Bus is a fixture for every daemon writer.

But not directly. It need not be a dependency on the part of the daemon
writer. In fact the daemon writer may now leave out deamonizing code
entirely if he or she wishes.  The systemd api is entirely optional for
the daemon to use.

> Guess what -- the unit files come with their own gotchas. For example,
> there is no simple way to convert an arbitrary pathname into an
> ExecStart entry of a unit file.
> 
> The .ini file format was a lousy choice. Why not choose JSON in this day
> and age?

But why json?  ini files are at least fairly human write-able and
read-able. Json is great, but not for this application. Thank goodness
they didn't choose xml. I never liked working with LaunchDaemon's plist
files or Solaris' service definition files.

> Yes, the SysV init system had become a jungle. Still, I wish it had been
> replaced with a rigorous protocol. The units should have been programs
> that comply with given contracts.

Yes there is certainly merit to this idea, if anyone could agree on a
protocol.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Bernd Nawothnig]

On 2016-12-05, Wildman wrote:
> And I am trying to write it without using external programs, where
> possible.

That is not the Unix way.

> I am a hobby programmer and I've been trying to learn python
> for a few months now.  The program is 'worthlessware' but it
> is a 'learning experience' for me.  

It looks for me like a worthless learning experience.

> A friend wrote a similar program in Bash script and I have been
> working on translating it to Python.

Stay with shell script for such tasks. It is never a good idea to
choose the programming language before closer evaluating the problem.




Bernd

-- 
No time toulouse
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 16:08:57 -0600, Tim Chase wrote:

> On 2016-12-05 14:58, Wildman via Python-list wrote:
>> I there a way to detect what the Linux runlevel is from
>> within a Python program?  I would like to be able to do
>> it without the use of an external program such as 'who'
>> or 'runlevel'.
> 
> You can use something like
> 
> https://gist.github.com/likexian/f9da722585036d372dca
> 
> to parse the /var/run/utmp contents.  Based on some source-code
> scrounging, it looks like you want the first field to be "1" for the
> "runlevel" account.  To extract the actual runlevel, you can take
> the PID value from the second column ("53" in my example here) and
> take it's integer value mod 256 (AKA "& 0xff") to get the character
> value.  So chr(int("53") & 0xff) returns "5" in my case, which is my
> runlevel.
> 
> Additional links I found helpful while searching:
> 
> https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
> https://github.com/garabik/python-utmp
> 
> -tkc

That is exactly the kind of thing I was looking for.  Thank you.
Now all I have to do is get it to work with Python3.

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: When will they fix Python _dbm?

[clvanwall]

will thid do?  John


Sent from my Galaxy Tab® A
 Original message From: justin walters 
 Date: 12/5/16  11:13 AM  (GMT-06:00) To: 
python-list@python.org Subject: Re: When will they fix Python _dbm? 
On Mon, Dec 5, 2016 at 6:45 AM, clvanwall  wrote:

> I have been a Perl programmer for 15+ years and decided to give Python a
> try.  My platform is windows and I installed the latest 3.5.2. Next I
> decided to convert a perl program that uses a ndbm database since according
> to the doc on python, it should be able to work with it.  Needless to say,
> I get: dbm.error: db type is dbm.ndbm, but the module is not available
> Searching on Python Bug Tracker shows _dbm missing back in 03-03-2012!
> That's a long time for a bug to be left open.
> John Van Walleghen
>
>
> Sent from my Galaxy Tab® A
> --
> https://mail.python.org/mailman/listinfo/python-list
>


Hi there,

Could you please post code that is giving you an error?
-- 
https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Michael Torrie]

On 12/05/2016 05:14 PM, Bernd Nawothnig wrote:
> On 2016-12-05, Wildman wrote:
>> And I am trying to write it without using external programs, where
>> possible.
> 
> That is not the Unix way.
> 
>> I am a hobby programmer and I've been trying to learn python
>> for a few months now.  The program is 'worthlessware' but it
>> is a 'learning experience' for me.  
> 
> It looks for me like a worthless learning experience.
> 
>> A friend wrote a similar program in Bash script and I have been
>> working on translating it to Python.
> 
> Stay with shell script for such tasks. It is never a good idea to
> choose the programming language before closer evaluating the problem.I

I think Python is a good choice for such a utility, but I agree it is
much better to rely on these external utilities as children to do the
platform-dependent work, rather than try to re-implement everything in
Python.  A long time ago I wrote a simple wrapper to Popen that would
run a command and return the standard out and standard error to me.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Nathan Ernst]

Rather than argue about what is/should be allowed by a filesystem, this
defines what is allowed on NTFS (default for modern Windows systems):
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

One can complain about whether or not something should be allowed, but,
you'd have to take that up with Microsoft (and I'll doubt you'll make a
convincing enough argument for them to change it).

What is allowed on linux may be defined by linux itself, and it may be
restricted by the filesystem type itself (I don't know).

Regards,
Nathan

On Mon, Dec 5, 2016 at 8:25 PM, Dennis Lee Bieber 
wrote:

> On Mon, 5 Dec 2016 20:55:41 +, BartC  declaimed the
> following:
>
> >This was a response to someone saying the wildcard param needs to be at
> >the end. There need be no such restriction if handled properly (ie. no
> >auto-expansion).
> >
> That only applies if there is no prefix indicating a command option
> from a file name.
>
> >but one of the files in the list is called "-lm", or some other option
>
> -lm is not a valid file name on the OS's that use - as an option
> prefix.
>
> >Without expansion, input is easy to parse: filespec, followed by
> >optional options. But with expansion, now you have to decide if a
> >particular argument is an option, or a filename.
> >
> And you do that using a delimiter character that is not valid in
> filenames. On Windows, file names can not have a /, and that is the
> character used by the command line interpreter to indicate an option
> follows. On UNIX, - is used as the delimiter of an option.
>
> --
> Wulfraed Dennis Lee Bieber AF6VN
> wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Steve D'Aprano]

On Tue, 6 Dec 2016 10:09 am, eryk sun wrote:

> On Mon, Dec 5, 2016 at 4:49 PM, Steve D'Aprano
>  wrote:
>>
>> You've never used cmd.com or command.exe? "The DOS prompt"?
> 
> The default Windows shell is "cmd.exe", and it's informally called the
> "Command Prompt", 

Thanks for the correction, I always mix up cmd/command . exe/com. I fear
this won't be the last time either -- I wish there was a good mnemonic for
which is which.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Steve D'Aprano]

On Tue, 6 Dec 2016 11:08 am, Michael Torrie wrote about systemd:

> I have yet to see any evidence of this Pyonguang situation.

Let me guess... you're running a single-user Linux box?

Fortunately, I've managed to avoid needing to personally interact with
systemd at all. But over the last year or so, I've had to listen to a
continual chorus of complaints from the sys admins I work with as they
struggle to adapt our code to the Brave New World of systemd.

Let me put it this way: one of our techs took it upon himself to migrate our
Python code base from Python 2.6 to 3.4, some tens of thousands of lines.
It took him half of one afternoon.

In comparison, migrating to systemd has given us nothing but headaches, and
invariably when we try asking for help on the main systemd IRC channel
we're told that we're wrong for wanting to do what we want to do.

Not just "systemd can't do that", but "you shouldn't do that".

Why not? We used to do it, and it is necessary for our application.

"Because its wrong."




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Nathan Ernst]

Ifyou're running on Windows 10, at least, you can soon purge that memory.
command.com doesn't exist (may never have existed on Win2k, XP, Vista, 7,
8, 8.1 or 10). If I try and run either "command" or "command.com" from
Win10, both say command cannot be found.

IIRC, command.com was a relic of Win9x running on top of DOS and was a
16-bit executable, so inherently crippled (and probably never support by
the NT kernel). Whereby cmd.exe coexisted but ran in a 32-bit context.

On Mon, Dec 5, 2016 at 8:44 PM, Steve D'Aprano 
wrote:

> On Tue, 6 Dec 2016 10:09 am, eryk sun wrote:
>
> > On Mon, Dec 5, 2016 at 4:49 PM, Steve D'Aprano
> >  wrote:
> >>
> >> You've never used cmd.com or command.exe? "The DOS prompt"?
> >
> > The default Windows shell is "cmd.exe", and it's informally called the
> > "Command Prompt",
>
> Thanks for the correction, I always mix up cmd/command . exe/com. I fear
> this won't be the last time either -- I wish there was a good mnemonic for
> which is which.
>
>
>
> --
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Nathan Ernst]

OT, but I'm curious, do they explain *why* it's wrong and give an
alternative, or just outright deride it as "the wrong way". I ask because
I've read similar complaints about the community around systemd, but as it
rarely affects me personally, I've never bothered to care.

On Mon, Dec 5, 2016 at 8:48 PM, Steve D'Aprano 
wrote:

> On Tue, 6 Dec 2016 11:08 am, Michael Torrie wrote about systemd:
>
> > I have yet to see any evidence of this Pyonguang situation.
>
> Let me guess... you're running a single-user Linux box?
>
> Fortunately, I've managed to avoid needing to personally interact with
> systemd at all. But over the last year or so, I've had to listen to a
> continual chorus of complaints from the sys admins I work with as they
> struggle to adapt our code to the Brave New World of systemd.
>
> Let me put it this way: one of our techs took it upon himself to migrate
> our
> Python code base from Python 2.6 to 3.4, some tens of thousands of lines.
> It took him half of one afternoon.
>
> In comparison, migrating to systemd has given us nothing but headaches, and
> invariably when we try asking for help on the main systemd IRC channel
> we're told that we're wrong for wanting to do what we want to do.
>
> Not just "systemd can't do that", but "you shouldn't do that".
>
> Why not? We used to do it, and it is necessary for our application.
>
> "Because its wrong."
>
>
>
>
> --
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 18:25:58 -0700, Michael Torrie wrote:

> I think Python is a good choice for such a utility, but I agree it is
> much better to rely on these external utilities as children to do the
> platform-dependent work, rather than try to re-implement everything in
> Python.  A long time ago I wrote a simple wrapper to Popen that would
> run a command and return the standard out and standard error to me.

My rational is that all Linux distros are not created equal.
One comes with one certain set of utilities and another can
have different ones.  I can't always depend on a given
utility being there.  And there is not way to know the
names of same utility across all distros.  This is especially
a problem when comparing .rpm with .deb based distros.

In cases where I have to use an external program, I mean in
cases where there is no apparent Python solution, I check
for the existence of that program and just skip over that
section of the code if it is not found.

BTW here is a link for the Bash script I mentioned in case
you would like to take a look at it.  The guy who wrote
it had only been learning Bash for a few months.  Not bad.
I have tried to tweak an interest in him for Python but
he sticks with Bash.  He says that is the best language
for programming on Linux and he is not interested in GUI
programming.

https://github.com/marek-novotny/linfo

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Michael Torrie]

On 12/05/2016 07:48 PM, Steve D'Aprano wrote:
> On Tue, 6 Dec 2016 11:08 am, Michael Torrie wrote about systemd:
> 
>> I have yet to see any evidence of this Pyonguang situation.
> 
> Let me guess... you're running a single-user Linux box?

No I've done it on servers that weren't single-user (mail and web
servers in particular).  Not sure what you're getting at there, or why
that's relevant.  All Linux machines are set up and run as multi-user
boxes anyway.  Even my laptop that only I use.

> Fortunately, I've managed to avoid needing to personally interact with
> systemd at all. But over the last year or so, I've had to listen to a
> continual chorus of complaints from the sys admins I work with as they
> struggle to adapt our code to the Brave New World of systemd.
> 
> Let me put it this way: one of our techs took it upon himself to migrate our
> Python code base from Python 2.6 to 3.4, some tens of thousands of lines.
> It took him half of one afternoon.
>
> In comparison, migrating to systemd has given us nothing but headaches, and
> invariably when we try asking for help on the main systemd IRC channel
> we're told that we're wrong for wanting to do what we want to do.

Well since I have no idea what you were trying to do I can't comment.

> Not just "systemd can't do that", but "you shouldn't do that".
> 
> Why not? We used to do it, and it is necessary for our application.

That sounds frustrating, but of course I've heard similar stories from
folks moving to Python 3. :)

> "Because its wrong."

I've yet to encounter any of those kind of problems your admins
apparently had.  Also systemd in no way prevents you from using init
scripts or even inetd services if you choose, so there's always a
fallback position.

I'll have to specifically ask my friend who works for Bluehost about any
systemd troubles next time I speak with her.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Tim Chase]

On 2016-12-05 18:26, Wildman via Python-list wrote:
> On Mon, 05 Dec 2016 16:08:57 -0600, Tim Chase wrote:
> 
> > On 2016-12-05 14:58, Wildman via Python-list wrote:
> >> I there a way to detect what the Linux runlevel is from
> >> within a Python program?  I would like to be able to do
> >> it without the use of an external program such as 'who'
> >> or 'runlevel'.
> > 
> > You can use something like
> > 
> > https://gist.github.com/likexian/f9da722585036d372dca
> > 
> > to parse the /var/run/utmp contents.  Based on some source-code
> > scrounging, it looks like you want the first field to be "1" for
> > the "runlevel" account.  To extract the actual runlevel, you can
> > take the PID value from the second column ("53" in my example
> > here) and take it's integer value mod 256 (AKA "& 0xff") to get
> > the character value.  So chr(int("53") & 0xff) returns "5" in my
> > case, which is my runlevel.
> > 
> > Additional links I found helpful while searching:
> > 
> > https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
> > https://github.com/garabik/python-utmp
> 
> That is exactly the kind of thing I was looking for.  Thank you.
> Now all I have to do is get it to work with Python3.

This works based on my poking at it in both Py2 and Py3:

  import struct
  from collections import namedtuple

  try:
basestring
  except NameError:
basestring = str

  UTMP = namedtuple("UTMP", [
  "ut_type", # Type of record
  "ut_pid", # PID of login process
  "ut_line", # Device name of tty - "/dev/"
  "ut_id", # Terminal name suffix, or inittab(5) ID
  "ut_user", # Username
  "ut_host", # Hostname for remote login, or kernel version for run-level 
messages
  "e_termination", # Process termination status
  "e_exit", # Process exit status
  "ut_session", # Session ID (getsid(2)), used for windowing
  "tv_sec", # Seconds
  "tv_usec", # Microseconds
  "ut_addr_v6a", # Internet address of remote host; IPv4 address uses just 
ut_addr_v6[0]
  "ut_addr_v6b", # Internet address of remote host; IPv4 address uses just 
ut_addr_v6[0]
  "ut_addr_v6c", # Internet address of remote host; IPv4 address uses just 
ut_addr_v6[0]
  "ut_addr_v6d", # Internet address of remote host; IPv4 address uses just 
ut_addr_v6[0]
  #"__unused", # Reserved for future use
  ])

  XTMP_STRUCT = "hi32s4s32s256shhiii20x"
  XTMP_STRUCT_SIZE = struct.calcsize(XTMP_STRUCT)

  # ut_types
  EMPTY = 0
  RUN_LVL = 1
  BOOT_TIME = 2
  OLD_TIME = 3
  NEW_TIME = 4
  INIT_PROCESS = 5 # Process spawned by "init"
  LOGIN_PROCESS = 6 # A "getty" process

  DEFAULT_UTMP = "/var/run/utmp"

  def parse_utmp(utmp_fname=DEFAULT_UTMP):
  with open(utmp_fname, "rb") as f:
  while True:
  bytes = f.read(XTMP_STRUCT_SIZE)
  if not bytes:
  break
  bits = struct.unpack(XTMP_STRUCT, bytes)
  bits = [
  bit.rstrip('\0') if isinstance(bit, basestring) else bit
  for bit
  in bits
  ]
  yield UTMP(*bits)

  def filter(ut_type, utmp_fname=DEFAULT_UTMP):
  for utmp in parse_utmp(utmp_fname):
  if utmp.ut_type == ut_type:
  yield utmp

  def get_runlevel(utmp_fname=DEFAULT_UTMP):
  return chr(next(filter(RUN_LVL, utmp_fname)).ut_pid & 0xFF)

  if __name__ == "__main__":
  print("Runlevel: %s" % get_runlevel())


-tkc






-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Michael Torrie]

On 12/05/2016 08:27 PM, Wildman via Python-list wrote:
> On Mon, 05 Dec 2016 18:25:58 -0700, Michael Torrie wrote:
> 
>> I think Python is a good choice for such a utility, but I agree it is
>> much better to rely on these external utilities as children to do the
>> platform-dependent work, rather than try to re-implement everything in
>> Python.  A long time ago I wrote a simple wrapper to Popen that would
>> run a command and return the standard out and standard error to me.
> 
> My rational is that all Linux distros are not created equal.
> One comes with one certain set of utilities and another can
> have different ones.  I can't always depend on a given
> utility being there.  And there is not way to know the
> names of same utility across all distros.  This is especially
> a problem when comparing .rpm with .deb based distros.

Well this is a problem regardless of which scripting language you choose
and the solutions will be the same.

> In cases where I have to use an external program, I mean in
> cases where there is no apparent Python solution, I check
> for the existence of that program and just skip over that
> section of the code if it is not found.

Sure. That's probably reasonable.

> 
> BTW here is a link for the Bash script I mentioned in case
> you would like to take a look at it.  The guy who wrote
> it had only been learning Bash for a few months.  Not bad.
> I have tried to tweak an interest in him for Python but
> he sticks with Bash.  He says that is the best language
> for programming on Linux and he is not interested in GUI
> programming.
> 
> https://github.com/marek-novotny/linfo

That all depends on what he's programming. For anything a user interacts
with, Bash is a pretty poor tool.  But Bash is really good at
shell-scripting system tasks.  It treats external commands as
first-class entities, and process control and I/O piping is integrated
into the syntax of the language.  On the other hand, Python is a good
language but it's not particularly well suited to shell scripting
(wasn't designed for that purpose), though it does have some facilities
like generators that make certain forms of system programming really
slick. In short they overlap in purpose, but they aren't good at the
same things.  However when a bash script gets too long, I'll often
switch to Python, using my run wrapper and generator filters to process
the output of external commands.

Personally for a script of this type, I'd probably stick with Bash
myself. Which by the way is what inxi is written in.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Michael Torrie]

On 12/05/2016 07:25 PM, Dennis Lee Bieber wrote:
> On Mon, 5 Dec 2016 20:55:41 +, BartC  declaimed
> the following:
> 
>> This was a response to someone saying the wildcard param needs to
>> be at the end. There need be no such restriction if handled
>> properly (ie. no auto-expansion).
>> 
> That only applies if there is no prefix indicating a command option 
> from a file name.
> 
>> but one of the files in the list is called "-lm", or some other
>> option
> 
> -lm is not a valid file name on the OS's that use - as an option 
> prefix.

"-" is perfectly valid in a filename on Linux. Getting apps to recognize
it as a filename and not an argument is another story. Convention is to
allow an argument "--" that tells the arg parser that everything
following that is not an argument but a parameter which may or may not
be a file--BartC seems stuck on this point, but parameters could be
anything from urls to string messages to numbers. They don't have to be
files and they in fact could begin with "/" if the program allowed it.

I argue that having convention used by programs and conventions sued by
shells instead of some kind of arbitrary OS-enforced scheme is
inherently more flexible and has its own consistency (everything is
explicit from any program's point of view).

>> Without expansion, input is easy to parse: filespec, followed by 
>> optional options. But with expansion, now you have to decide if a 
>> particular argument is an option, or a filename.
>> 
> And you do that using a delimiter character that is not valid in 
> filenames. On Windows, file names can not have a /, and that is the 
> character used by the command line interpreter to indicate an option 
> follows. On UNIX, - is used as the delimiter of an option.

What is valid in a filename is not relevant here.  No one says
command-line parameters are file names if they aren't an option or
argument.  What a parameter means is up to the application (convention).
 Windows disallows / in a filename.  Linux happens to also.  But Linux
does allow '-' in a filename.  Any character can be used in a parameter
to a program except \0, from the program's point of view.

Back to shell expansion, if I had a file named "-atest.txt" and I passed
an argument to a command like this:

command -a*

The shell would in fact expand that to "-atest.txt" much to BartC's
consternation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 20:46:22 -0700, Michael Torrie wrote:

> On 12/05/2016 08:27 PM, Wildman via Python-list wrote:
>> On Mon, 05 Dec 2016 18:25:58 -0700, Michael Torrie wrote:
>> 
>>> I think Python is a good choice for such a utility, but I agree it is
>>> much better to rely on these external utilities as children to do the
>>> platform-dependent work, rather than try to re-implement everything in
>>> Python.  A long time ago I wrote a simple wrapper to Popen that would
>>> run a command and return the standard out and standard error to me.
>> 
>> My rational is that all Linux distros are not created equal.
>> One comes with one certain set of utilities and another can
>> have different ones.  I can't always depend on a given
>> utility being there.  And there is not way to know the
>> names of same utility across all distros.  This is especially
>> a problem when comparing .rpm with .deb based distros.
> 
> Well this is a problem regardless of which scripting language you choose
> and the solutions will be the same.

It is a problem only if you depend on the utility.

> Personally for a script of this type, I'd probably stick with Bash
> myself.

In most cases I would agree with that, but, in this case my
goal is learning Python.  I did write a couple of programs
with Bash several months ago to learn a little about it.
One will take an image and convert it into an X-Face header
and the other will take an image and convert it into a Face
header.  I later wrote GUI versions of the programs with
Python and Tkinter.

BTW, I don't depend on programming for a living.  I would be
in bad shape if I did.  It is a hobby that I greatly enjoy.
And, being in my later years, it keeps my mind sharp(er).

> Which by the way is what inxi is written in.

Yes, I was aware of that.  It's over 12,000 lines!

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Wildman via Python-list]

On Mon, 05 Dec 2016 21:42:52 -0600, Tim Chase wrote:

> On 2016-12-05 18:26, Wildman via Python-list wrote:
>> On Mon, 05 Dec 2016 16:08:57 -0600, Tim Chase wrote:
>> 
>> > On 2016-12-05 14:58, Wildman via Python-list wrote:
>> >> I there a way to detect what the Linux runlevel is from
>> >> within a Python program?  I would like to be able to do
>> >> it without the use of an external program such as 'who'
>> >> or 'runlevel'.
>> > 
>> > You can use something like
>> > 
>> > https://gist.github.com/likexian/f9da722585036d372dca
>> > 
>> > to parse the /var/run/utmp contents.  Based on some source-code
>> > scrounging, it looks like you want the first field to be "1" for
>> > the "runlevel" account.  To extract the actual runlevel, you can
>> > take the PID value from the second column ("53" in my example
>> > here) and take it's integer value mod 256 (AKA "& 0xff") to get
>> > the character value.  So chr(int("53") & 0xff) returns "5" in my
>> > case, which is my runlevel.
>> > 
>> > Additional links I found helpful while searching:
>> > 
>> > https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
>> > https://github.com/garabik/python-utmp
>> 
>> That is exactly the kind of thing I was looking for.  Thank you.
>> Now all I have to do is get it to work with Python3.
> 
> This works based on my poking at it in both Py2 and Py3:

That works perfectly.  I owe you a big thanks.  That was a
lot of work and time on your part.  I really appreciate it.

-- 
 GNU/Linux user #557453
The cow died so I don't need your bull!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Steven D'Aprano]

On Tuesday 06 December 2016 14:57, Michael Torrie wrote:

> "-" is perfectly valid in a filename on Linux. Getting apps to recognize
> it as a filename and not an argument is another story. Convention is to
> allow an argument "--" that tells the arg parser that everything
> following that is not an argument but a parameter which may or may not
> be a file

Another useful trick is to use a relative or absolute path to refer to a 
filename that begins with a dash:

./-foo

is unambiguously the file called "-foo" in the current directory, not the -f 
option.

This doesn't help when it comes to arguments which don't refer to filenames, 
hence the -- tradition.


> --BartC seems stuck on this point, but parameters could be
> anything from urls to string messages to numbers. They don't have to be
> files and they in fact could begin with "/" if the program allowed it.

Indeed.

The Unix shells are optimized for a particular use-case: system administration. 
For that, the globbing conventions etc are pretty close to optimal, but of 
course they're not the only conventions possible. Unix shells recognise this 
and allow you to escape metacharacters, and even turn off globbing altogether. 
Another alternative would be to eschew the use of a single command line and 
build your own scripting mini-language with its own conventions. That's 
especially useful if your primary use-case is to invoke your program many 
times, rather than just once.

E.g. the Postgresql interactive interpreter allows you to run multiple queries 
interactively without worrying about the shell:

https://www.postgresql.org/docs/current/static/tutorial-accessdb.html


There's no doubt that Bart has a legitimate use-case: he wants his input to be 
fed directly to his program with no pre-processing. (At least, no globbing: if 
he has given his opinion on environment variable expansion, I missed it.) Fair 
enough. Its easy to escape wildcards, but perhaps there should be an easy way 
to disable *all* pre-processing for a single command.

The fact that there is no easy, well-known way to do so indicates just how 
unusual Bart's use-case is. Linux and Unix sys admins are really good at 
scratching their own itches, and believe me, if there was widespread wish to 
disable pre-processing for a single command, after 40-odd years of Unix the 
majority of shells would support it.



-- 
Steven
"Ever since I learned about confirmation bias, I've been seeing 
it everywhere." - Jon Ronson

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Gregory Ewing]

BartC wrote:

On 05/12/2016 19:29, Michael Torrie wrote:


On 12/05/2016 11:50 AM, BartC wrote:




So how do I do:

gcc *.c -lm

The -lm needs to go at the end.

Presumably it now needs to check each parameter seeing if it resembles a
file more than it does an option? And are options automatically taken
care of, or is that something that, unlike the easier wildcards, need to
be processed explicitly?



This was a response to someone saying the wildcard param needs to be at 
the end. There need be no such restriction if handled properly (ie. no 
auto-expansion).


But a similar example, suppose a syntax is:

  appl *.* [options]

but one of the files in the list is called "-lm", or some other option 
that can do things the user didn't want (with gcc, -lm is harmless).


Without expansion, input is easy to parse: filespec, followed by 
optional options. But with expansion, now you have to decide if a 
particular argument is an option, or a filename.


And if there's an error in an option, you may have to abort, which means 
throwing away that list of files which, in some cases, can run into 
millions.



Not having glob expansion be in individual programs makes things much
more explicit, from the program's point of view.  It provides an
interface that takes filename(s) and you provide them, either explicitly
(via popen with no shell), or you can do it implicitly but in an
interactive way via the shell using expansion.  Personal preference but
I believe it's a much better way because it's explicit from the
program's point of view as there's no question about the program's 
behavior.


Different strokes for different folks as they say.



I must have given ten or twenty scenarios where such auto-expansion is 
problematical. And yet people are still in denial. It's been in Unix for 
decades therefore there's nothing wrong with it!


But maybe people simply avoid all the situations that cause problems. 
Interfaces are specified in a certain manner, given that input can be 
any long stream of filenames and/or options with no order and no 
positional significance. Non-file parameters that use ? or * are 
prohibited. You can't do the equivalent of:


 >DIR *.b *.c

And get a list of *.b files, with a heading and summary lines, then a 
list of *.c files with its own heading and summary. It would be one 
monolithic list.


So everyone is working around the restrictions and the problems. Which 
is exactly what I would have to do.


That doesn't change the fact that the Windows approach is considerably 
more flexible and allowing more possibilities.



--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Larry Hudson via Python-list]

On 12/05/2016 06:51 PM, Nathan Ernst wrote:

IIRC, command.com was a relic of Win9x running on top of DOS and was a
16-bit executable, so inherently crippled (and probably never support by
the NT kernel). Whereby cmd.exe coexisted but ran in a 32-bit context.


I know my 79-year-old memory is definitely subject to "senior moments" and not too reliable, but 
IIRC it was Windows prior to 9x (Win 3 and earlier) that were 16 bit and ran on top of DOS. 
Win95 was the first 32 bit version and was independent from DOS.


--
 -=- Larry -=-
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Gregory Ewing]

BartC wrote:

But a similar example, suppose a syntax is:

  appl *.* [options]


I would be disappointed by such a syntax. What if I want
to operate on two or more files with unrelated names? With
that syntax, I can't list them explicitly in the one command.

To make that possible, the syntax would have to be

   appl filespec... [options]

i.e. allow an arbitrary number of filespecs followed by
options -- requiring the command line to be scanned looking
for options anyway.

And if there's an error in an option, you may have to abort, which means 
throwing away that list of files which, in some cases, can run into 
millions.


This "millions of files" thing seems to be an imaginary
monster you've invented to try to scare people. I claim
that, to a very good approximation, it doesn't exist.
I've never encountered a directory containing a million
files, and if such a thing existed, it would be pathological
in enough other ways to make it a really bad idea.

But maybe people simply avoid all the situations that cause problems. 
Interfaces are specified in a certain manner, given that input can be 
any long stream of filenames and/or options with no order and no 
positional significance.


Exactly. The programs are designed with knowledge of the
way shells behave and are typically used.


You can't do the equivalent of:

 >DIR *.b *.c

And get a list of *.b files, with a heading and summary lines, then a 
list of *.c files with its own heading and summary.


Not with *that particular syntax*. You would need to
design the interface of a program to do that some other
way.

That doesn't change the fact that the Windows approach is considerably 
more flexible and allowing more possibilities.


At the expense of having shells with less powerful
facilities, and more inconsistent behaviour between
different programs.

One isn't better than the other. There are tradeoffs.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Gregory Ewing]

Dennis Lee Bieber wrote:

-lm is not a valid file name on the OS's that use - as an option
prefix.


It's not invalid -- you can create a file called -lm
on a unix system if you want, you just have to be a bit
sneaky about how you refer to it:

% echo foo > ./-lm
% ls
-lm
% cat ./-lm
foo

Sane people normally refrain from using such file names,
however, because of the hassle of dealing with them.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 2.7.12 on Linux behaving differently than on Windows

[Larry Hudson via Python-list]

On 12/05/2016 10:50 AM, BartC wrote:


And just what ARE A, C, and D?


It doesn't matter, and is not the concern of the shell. It should restrict 
itself to the basic
parsing that may be necessary when parameters are separated by white-space and 
commas, if a
parameter can contain white-space or commas. That usually involves surrounding 
the parameter
with quotes.

One would be very annoyed if, reading a CSV file, where each of N values on a 
line correspond to
a field of record, if one entry of "?LBC" expanded itself to a dozen entries, 
screwing
everything up.


Now you're suggesting the _shell_ is going to read and process a CVS file???


Shell command line processing shouldn't be attempting anything more than that.

I get awfully tired of your constant pontificating about your opinions.  I know they're _VERY_ 
strongly held beliefs on your part, but...  they are ONE person's opinions and are no more than 
opinions and they ain't gonna change nothin', no how, no way, not ever.


[Sorry, I'm in a bad mood today and just had to let off a little steam...]

--
 -=- Larry -=-
--
https://mail.python.org/mailman/listinfo/python-list

Re: Detect Linux Runlevel

[Marko Rauhamaa]

Michael Torrie :

> On 12/05/2016 04:37 PM, Marko Rauhamaa wrote:
>> Unfortunately, I am not wholly impressed by the end result. Mogadishu
>> has been replaced by Pyongyang. Some age-old Unix principles have been
>> abandoned without clear justification. For example, I was appalled to
>> find out that a systemd unit can be configured to react on the exit
>> status of a child process of a daemon.
>
> I have yet to see any evidence of this Pyonguang situation.
>
> What is wrong with reacting to the exit status of a daemon's child
> process?

A basic black-box principle is violated. It is surprising, at least.

If I launch a child process, wait it out and ignore its exit status
code, I would think the exit status is meaningless. Not so, because the
Eye is watching.

>> Also, now D-Bus is a fixture for every daemon writer.
>
> But not directly. It need not be a dependency on the part of the
> daemon writer. In fact the daemon writer may now leave out deamonizing
> code entirely if he or she wishes. The systemd api is entirely
> optional for the daemon to use.

Systemd comes with dozens of legacy modes, and it is difficult to learn
what is systemd's "native" daemon type. However, it is evident that
"Type=notify" is it, meaning the daemon communicates with systemd
explicitly (https://www.freedesktop.org/software/systemd/man/sd_notify.html>).

That is not necessarily a bad idea. Daemons have traditionally been
lousy at communicating their statuses appropriately. It's just that this
major requirement should be declared openly.

>> The .ini file format was a lousy choice. Why not choose JSON in this
>> day and age?
>
> But why json?

It comes with simple, rigorous, universal syntax. The .ini files don't.
That's why the unit files have brittle ad-hoc syntax.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list