Re: [Tutor] subprocess.Popen basics

2014-10-28 Thread Adam Jensen
Update:

On 10/27/2014 09:50 PM, Adam Jensen wrote:
> What's weird is that I have two different python3.4 installations on
> this CentOS-6.5 machine and both have the same behavior (script hangs
> until Ctrl+C).
> 
> I built this one (/opt/bin/python3.4) from source:
...
> But this one (~/anaconda3/bin/python3.4) was a binary distribution (if I
> recall correctly):

This ^ undermines the "build problem" theory. I found a little blurb in
the documentation about bufsize:

-
bufsize will be supplied as the corresponding argument to the open()
function when creating the stdin/stdout/stderr pipe file objects:

* 0 means unbuffered (read and write are one system call and can return
short)
* 1 means line buffered (only usable if universal_newlines=True i.e., in
a text mode)
* any other positive value means use a buffer of approximately that size
* negative bufsize (the default) means the system default of
io.DEFAULT_BUFFER_SIZE will be used.
-

Some tinkering:

-
#!/usr/bin/env python3.4

import subprocess as sp

parrot = sp.Popen(['./parrot.sh'], bufsize=0,
stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE,
universal_newlines=True, start_new_session=True)

parrot.stdin.write('Pushing up daisies.\n')
print('Parrot said: ', parrot.stdout.readline())
parrot.kill()
-

And I get these results (on CentOS-6.5-x86):

| bufsize | results |
|-+-|
| default | hangs   |
|  -1 | hangs   |
|   0 | works   |
|   1 | hangs   |
| >=2 | works   |

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] subprocess.Popen basics

2014-10-28 Thread Alan Gauld

On 28/10/14 15:31, Adam Jensen wrote:


-
bufsize will be supplied as the corresponding argument to the open()
function when creating the stdin/stdout/stderr pipe file objects:



And I get these results (on CentOS-6.5-x86):

| bufsize | results |
|-+-|
| default | hangs   |
|  -1 | hangs   |
|   0 | works   |
|   1 | hangs   |
| >=2 | works   |


I tried -1 and 1 on my Lubuntu and it still works fine.
Definitely weird, it begins to look like a CentOS build issue
but what is CentOS doing different to Lubuntu/Suse/OpenBSD etc?

From memory CentOS is basically a free version of Red Hat
Enterprise Linux? I can't see why that would be significantly
different in stdin/out behaviour.

weirder and weirder

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] subprocess.Popen basics

2014-10-28 Thread Adam Jensen

On 10/28/2014 02:32 PM, Alan Gauld wrote:
> I tried -1 and 1 on my Lubuntu and it still works fine.
> Definitely weird, it begins to look like a CentOS build issue
> but what is CentOS doing different to Lubuntu/Suse/OpenBSD etc?
> 
> From memory CentOS is basically a free version of Red Hat
> Enterprise Linux? I can't see why that would be significantly
> different in stdin/out behaviour.
> 
> weirder and weirder

Yes, CentOS is a RHEL clone. Scientific-Linux is another RHEL clone
(assembled and maintained by several of the national labs). I'm fairly
certain that Python is in heavy use at the labs so it seems especially
weird that there is something so wonky going on with it on that
platform. This is what I've discovered so far:

| | CentOS-6.5   | OpenBSD-5.5  | DragonFly-3.8.2 |
| bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3|
|-+--+--+-|
| default | hangs| works| works   |
| -1  | hangs| works| works   |
| 0   | works| works| works   |
| 1   | hangs| works| works   |
| >=2 & <(# of bytes) | works| works| works   |
| >=(# of bytes)  | hangs| works| works   |

(The number of bytes in 'Pushing up daisies.\n' is 20).

io.DEFAULT_BUFFER_SIZE == 8192 on all three platforms.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] subprocess.Popen basics

2014-10-28 Thread Alan Gauld

On 28/10/14 19:23, Adam Jensen wrote:


platform. This is what I've discovered so far:

| | CentOS-6.5   | OpenBSD-5.5  | DragonFly-3.8.2 |
| bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3|
|-+--+--+-|
| default | hangs| works| works   |
| -1  | hangs| works| works   |
| 0   | works| works| works   |
| 1   | hangs| works| works   |
| >=2 & <(# of bytes) | works| works| works   |
| >=(# of bytes)  | hangs| works| works   |



Looks like you might be best trying this on a CentOS forum/list.
Its not exactly a mainstream distro so the number of folks on this list 
who can help is probably limited and even on the main Python list there 
probably are only a handful of CentOS users. And it looks increasingly 
like a CentOS configuration issue.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] subprocess.Popen basics

2014-10-28 Thread Todd
Centos has SELinux enabled by default.  I dont know if SELinux is causing
your problem, but it is always worth looking at.

SELinux can keep a process from accessing files or executing another
process.

Try temporarily disabling  SELinux by running setenforce=0 as root.  Then
see if python does what you expect.



On Tue, Oct 28, 2014 at 3:04 PM, Alan Gauld 
wrote:

> On 28/10/14 19:23, Adam Jensen wrote:
>
>  platform. This is what I've discovered so far:
>>
>> | | CentOS-6.5   | OpenBSD-5.5  | DragonFly-3.8.2 |
>> | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3|
>> |-+--+--+-|
>> | default | hangs| works| works   |
>> | -1  | hangs| works| works   |
>> | 0   | works| works| works   |
>> | 1   | hangs| works| works   |
>> | >=2 & <(# of bytes) | works| works| works   |
>> | >=(# of bytes)  | hangs| works| works   |
>>
>>
> Looks like you might be best trying this on a CentOS forum/list.
> Its not exactly a mainstream distro so the number of folks on this list
> who can help is probably limited and even on the main Python list there
> probably are only a handful of CentOS users. And it looks increasingly like
> a CentOS configuration issue.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.flickr.com/photos/alangauldphotos
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] subprocess.Popen basics

2014-10-28 Thread Adam Jensen

On 10/28/2014 04:27 PM, Todd wrote:
> Centos has SELinux enabled by default.  I dont know if SELinux is
> causing your problem, but it is always worth looking at.
> 
> SELinux can keep a process from accessing files or executing another
> process. 
> 
> Try temporarily disabling  SELinux by running setenforce=0 as root. 
> Then see if python does what you expect.

Yep, that occurred to me too. Earlier today I set 'SELINUX=disabled' in
/etc/selinux/config and did a 'sudo touch /.autorelabel' then rebooted.
No Joy, same behavior from subprocess.Popen().



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


[Tutor] Would somebody kindly...

2014-10-28 Thread Clayton Kirkwood
Explain this double speak(>:

[pair for pair in values if key == pair[0]]

 

I understand the 'for pair in values'. I assume the first 'pair' creates the
namespace (although I am not sure how Python knows it's a tuple yet). I
think the outer [] make the line a comprehension ( If so, I don't seem to
get how/why a comprehension works), and probably make the line either a
tuple or list. The comprehension seems to act as a subroutine or macro( yes
I know those two words are meaningless in Python). I definitely am missing
the full meaning of the if statement. The example comes courtesy of Alan.

 

Thanks,

 

Clayton

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Danny Yoo
On Tue, Oct 28, 2014 at 4:13 PM, Clayton Kirkwood  wrote:
> Explain this double speak(>:
>
> [pair for pair in values if key == pair[0]]

Hi Clayton,

Here is a rewording of that expression to an approximately equivalent statement:

###
some_list = []
for pair in values:
if key == pair[0]:
some_list.append(pair)
###
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Alan Gauld

On 28/10/14 23:13, Clayton Kirkwood wrote:

Explain this double speak(>:

[pair for pair in values if key == pair[0]]


A list comprehension is a specific form of a more general
construction called a generator expression. Its specific
in that it creates a list, hence the [] around it.
The general shape of a generator expression is:

 for value in  if 

and the comprehension version looks like

[  for value in  if  ]

The if condition part is optional in both cases.

This is syntactically equivalent to

aList = []
for value in iterator:
   if condition:
  aList.append(result expression)

Let's look at a specific example using literals:

[item for item in  [1,2,3,4,5] if item > 3]

The effect of which is the list [4,5]

So it acts as a filter that extracts the items in the inner
list that are greater than 3 (the if condition).

In your case I had proposed using a list of tuples(pairs)
like so:

values = [(1,2),(2,3),(3,4),(4,5)]

So the expression:

[pair for pair in values if 2 in pair]

will return [(1,2),(2,3)]

and the expression

[pair for pair in values if 2 == pair[0]]

will return [(2,3)] - the only pair with 2 in the first position

Now, coming back to your original question:

[pair for pair in values if key == pair[0]]

this returns the tuples that have the first element equal
to key - whatever the variable key happens to be.

PS.
For bonus points you can research how generator expressions
can encapsulate nested loops, but that's probably a step too
far for just now!

HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Alan Gauld

On 28/10/14 23:13, Clayton Kirkwood wrote:


line either a tuple or list. The comprehension seems to act as a
subroutine or macro


In a sense yes, that's true.

You can find another explanation of list comprehensions and other 
related functions in my tutorial (see .sig) in the Functional 
Programming topic. It has several more examples.



--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Clayton Kirkwood
Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had 
the some_list and pair the same name, presumably creating a temporary tuple and 
when the loop is done, the temporary replaces the original.

Thanks

Clayton 

!-Original Message-
!From: Danny Yoo [mailto:d...@hashcollision.org]
!Sent: Tuesday, October 28, 2014 4:23 PM
!To: Clayton Kirkwood
!Cc: Python Tutor Mailing List
!Subject: Re: [Tutor] Would somebody kindly...
!
!On Tue, Oct 28, 2014 at 4:13 PM, Clayton Kirkwood 
!wrote:
!> Explain this double speak(>:
!>
!> [pair for pair in values if key == pair[0]]
!
!Hi Clayton,
!
!Here is a rewording of that expression to an approximately equivalent
!statement:
!
!###
!some_list = []
!for pair in values:
!if key == pair[0]:
!some_list.append(pair)
!###



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Alan Gauld

On 29/10/14 01:02, Clayton Kirkwood wrote:

Ah, Alan sent an answer also, but this one answers the last tidbit.
Alan had the some_list and pair the same name, presumably creating

> a temporary tuple and when the loop is done, the temporary

replaces the original.


Nope. If you re-read my post you'll seee that my examples are just 
expressions, they don;t assign the result. Dannys code is actually the 
equivalent of:


some_list = [pair for pair in values if key == pair[0]]

some_list is an arbitrary name Danny used to represent the list
that is built up by the comprehension. It doesn't actually have
a name in the real comprehension. The name 'pair' in my code is
used exactly as in Danny's 'for' loop below.


!###
!some_list = []
!for pair in values:
!if key == pair[0]:
!some_list.append(pair)
!###



There is no temporary tuples being created the list is built up
inside the comprehension as the for part executes in the same
way as in the expanded version.

There is no magic here it's just a short-cut notation for the
expanded loop (and it usually executes a bit faster).

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Dave Angel

> 
 Explain this double speak(>:
>  [pair for pair in values if key == pair[0]]
 
>  I understand the ‘for pair in values’. I assume the first
>  ‘pair’ creates the namespace

The namespace question depends on the version of Python. Python
 2.x does not do any scoping.

But in version 3.x, the variable pair will go away.

So please tell us the version you're asking about. 

The list comprehension always creates a list, not a tuple. Pair is
 a tuple if and only if the corresponding element of values is a
 tuple. In fact there's no requirement that those elements of
 values have the same type. Of course if they're not of length 2,
 then pair is a lousy name. And if one of them is not
 subscriptable, then the if expression will blow up.


-- 
DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Cameron Simpson

On 28Oct2014 18:02, Clayton Kirkwood  wrote:

!> Explain this double speak(>:
!>
Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had 
the some_list and pair the same name, presumably creating a temporary tuple and 
when the loop is done, the temporary replaces the original.


Let me try a less wordy diagram. You will need to be displaying in a constant 
width font :-)


  [ pair for pair in values if key == pair[0] ]
-- the expression that accrues in the resulting list
 -- the loop variable, taken from the loop source values
 ^^-- the loop source values
   ^^-- condition for including the
   expression in the resulting list

So that first "pair" could be any expression, it is almost only coincidence 
that it is the same as the loop variable. It is the same in this case because 
this is the idiomatic way to select particular values form an existing list.


If we'd wanted the new list to contain double the original values we'd write:

  [ pair*2 for pair in values if key == pair[0] ]

Cheers,
Cameron Simpson 

ERROR 155 - You can't do that.  - Data General S200 Fortran error code list
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Clayton Kirkwood


!-Original Message-
!From: Cameron Simpson [mailto:c...@zip.com.au]
!Sent: Tuesday, October 28, 2014 6:31 PM
!To: Clayton Kirkwood
!Cc: 'Danny Yoo'; 'Python Tutor Mailing List'
!Subject: Re: [Tutor] Would somebody kindly...
!
!On 28Oct2014 18:02, Clayton Kirkwood  wrote:
!>!> Explain this double speak(>:
!>!>
!>Ah, Alan sent an answer also, but this one answers the last tidbit.
!Alan had the some_list and pair the same name, presumably creating a
!temporary tuple and when the loop is done, the temporary replaces the
!original.
!
!Let me try a less wordy diagram. You will need to be displaying in a
!constant width font :-)
!
!   [ pair for pair in values if key == pair[0] ]
! -- the expression that accrues in the resulting list
!  -- the loop variable, taken from the loop source
!values
!  ^^-- the loop source values
!^^-- condition for including
!the
!expression in the resulting
!list
!
!So that first "pair" could be any expression, it is almost only
!coincidence that it is the same as the loop variable. It is the same in
!this case because this is the idiomatic way to select particular values
!form an existing list.
!
!If we'd wanted the new list to contain double the original values we'd
!write:
!
!   [ pair*2 for pair in values if key == pair[0] ]

Ok, I am somewhat confused again. In the original example up above, it
appears that the pair list or tuple gets overridden. In this one right
above, once again, the list gets overwritten again, but what is being
doubled?, each part of the tuple?

Clayton

!
!Cheers,
!Cameron Simpson 
!
!ERROR 155 - You can't do that.  - Data General S200 Fortran error code
!list



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Would somebody kindly...

2014-10-28 Thread Clayton Kirkwood


!-Original Message-
!From: Tutor [mailto:tutor-bounces+crk=godblessthe...@python.org] On
!Behalf Of Dave Angel
!Sent: Tuesday, October 28, 2014 6:34 PM
!To: tutor@python.org
!Subject: Re: [Tutor] Would somebody kindly...
!
!
!>
! Explain this double speak(>:
!>  [pair for pair in values if key == pair[0]]
!
!>  I understand the ‘for pair in values’. I assume the first  ‘pair’
!> creates the namespace
!
!The namespace question depends on the version of Python. Python  2.x
!does not do any scoping.
!
!But in version 3.x, the variable pair will go away.
!
!So please tell us the version you're asking about.

I am using 3.4.1.

!
!The list comprehension always creates a list, not a tuple. Pair is  a
!tuple if and only if the corresponding element of values is a  tuple. In
!fact there's no requirement that those elements of  values have the same
!type. Of course if they're not of length 2,  then pair is a lousy name.
!And if one of them is not  subscriptable, then the if expression will
!blow up.
!
!
!--
!DaveA
!
!___
!Tutor maillist  -  Tutor@python.org
!To unsubscribe or change subscription options:
!https://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor