Re: [Tutor] subprocess.Popen basics
Update: On 10/27/2014 09:50 PM, Adam Jensen wrote: > What's weird is that I have two different python3.4 installations on > this CentOS-6.5 machine and both have the same behavior (script hangs > until Ctrl+C). > > I built this one (/opt/bin/python3.4) from source: ... > But this one (~/anaconda3/bin/python3.4) was a binary distribution (if I > recall correctly): This ^ undermines the "build problem" theory. I found a little blurb in the documentation about bufsize: - bufsize will be supplied as the corresponding argument to the open() function when creating the stdin/stdout/stderr pipe file objects: * 0 means unbuffered (read and write are one system call and can return short) * 1 means line buffered (only usable if universal_newlines=True i.e., in a text mode) * any other positive value means use a buffer of approximately that size * negative bufsize (the default) means the system default of io.DEFAULT_BUFFER_SIZE will be used. - Some tinkering: - #!/usr/bin/env python3.4 import subprocess as sp parrot = sp.Popen(['./parrot.sh'], bufsize=0, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE, universal_newlines=True, start_new_session=True) parrot.stdin.write('Pushing up daisies.\n') print('Parrot said: ', parrot.stdout.readline()) parrot.kill() - And I get these results (on CentOS-6.5-x86): | bufsize | results | |-+-| | default | hangs | | -1 | hangs | | 0 | works | | 1 | hangs | | >=2 | works | ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.Popen basics
On 28/10/14 15:31, Adam Jensen wrote: - bufsize will be supplied as the corresponding argument to the open() function when creating the stdin/stdout/stderr pipe file objects: And I get these results (on CentOS-6.5-x86): | bufsize | results | |-+-| | default | hangs | | -1 | hangs | | 0 | works | | 1 | hangs | | >=2 | works | I tried -1 and 1 on my Lubuntu and it still works fine. Definitely weird, it begins to look like a CentOS build issue but what is CentOS doing different to Lubuntu/Suse/OpenBSD etc? From memory CentOS is basically a free version of Red Hat Enterprise Linux? I can't see why that would be significantly different in stdin/out behaviour. weirder and weirder -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.Popen basics
On 10/28/2014 02:32 PM, Alan Gauld wrote: > I tried -1 and 1 on my Lubuntu and it still works fine. > Definitely weird, it begins to look like a CentOS build issue > but what is CentOS doing different to Lubuntu/Suse/OpenBSD etc? > > From memory CentOS is basically a free version of Red Hat > Enterprise Linux? I can't see why that would be significantly > different in stdin/out behaviour. > > weirder and weirder Yes, CentOS is a RHEL clone. Scientific-Linux is another RHEL clone (assembled and maintained by several of the national labs). I'm fairly certain that Python is in heavy use at the labs so it seems especially weird that there is something so wonky going on with it on that platform. This is what I've discovered so far: | | CentOS-6.5 | OpenBSD-5.5 | DragonFly-3.8.2 | | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3| |-+--+--+-| | default | hangs| works| works | | -1 | hangs| works| works | | 0 | works| works| works | | 1 | hangs| works| works | | >=2 & <(# of bytes) | works| works| works | | >=(# of bytes) | hangs| works| works | (The number of bytes in 'Pushing up daisies.\n' is 20). io.DEFAULT_BUFFER_SIZE == 8192 on all three platforms. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.Popen basics
On 28/10/14 19:23, Adam Jensen wrote: platform. This is what I've discovered so far: | | CentOS-6.5 | OpenBSD-5.5 | DragonFly-3.8.2 | | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3| |-+--+--+-| | default | hangs| works| works | | -1 | hangs| works| works | | 0 | works| works| works | | 1 | hangs| works| works | | >=2 & <(# of bytes) | works| works| works | | >=(# of bytes) | hangs| works| works | Looks like you might be best trying this on a CentOS forum/list. Its not exactly a mainstream distro so the number of folks on this list who can help is probably limited and even on the main Python list there probably are only a handful of CentOS users. And it looks increasingly like a CentOS configuration issue. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.Popen basics
Centos has SELinux enabled by default. I dont know if SELinux is causing your problem, but it is always worth looking at. SELinux can keep a process from accessing files or executing another process. Try temporarily disabling SELinux by running setenforce=0 as root. Then see if python does what you expect. On Tue, Oct 28, 2014 at 3:04 PM, Alan Gauld wrote: > On 28/10/14 19:23, Adam Jensen wrote: > > platform. This is what I've discovered so far: >> >> | | CentOS-6.5 | OpenBSD-5.5 | DragonFly-3.8.2 | >> | bufsize | Python-3.4.1 | Python-3.3.2 | Python-3.3.3| >> |-+--+--+-| >> | default | hangs| works| works | >> | -1 | hangs| works| works | >> | 0 | works| works| works | >> | 1 | hangs| works| works | >> | >=2 & <(# of bytes) | works| works| works | >> | >=(# of bytes) | hangs| works| works | >> >> > Looks like you might be best trying this on a CentOS forum/list. > Its not exactly a mainstream distro so the number of folks on this list > who can help is probably limited and even on the main Python list there > probably are only a handful of CentOS users. And it looks increasingly like > a CentOS configuration issue. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.flickr.com/photos/alangauldphotos > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] subprocess.Popen basics
On 10/28/2014 04:27 PM, Todd wrote: > Centos has SELinux enabled by default. I dont know if SELinux is > causing your problem, but it is always worth looking at. > > SELinux can keep a process from accessing files or executing another > process. > > Try temporarily disabling SELinux by running setenforce=0 as root. > Then see if python does what you expect. Yep, that occurred to me too. Earlier today I set 'SELINUX=disabled' in /etc/selinux/config and did a 'sudo touch /.autorelabel' then rebooted. No Joy, same behavior from subprocess.Popen(). ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Would somebody kindly...
Explain this double speak(>: [pair for pair in values if key == pair[0]] I understand the 'for pair in values'. I assume the first 'pair' creates the namespace (although I am not sure how Python knows it's a tuple yet). I think the outer [] make the line a comprehension ( If so, I don't seem to get how/why a comprehension works), and probably make the line either a tuple or list. The comprehension seems to act as a subroutine or macro( yes I know those two words are meaningless in Python). I definitely am missing the full meaning of the if statement. The example comes courtesy of Alan. Thanks, Clayton ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
On Tue, Oct 28, 2014 at 4:13 PM, Clayton Kirkwood wrote: > Explain this double speak(>: > > [pair for pair in values if key == pair[0]] Hi Clayton, Here is a rewording of that expression to an approximately equivalent statement: ### some_list = [] for pair in values: if key == pair[0]: some_list.append(pair) ### ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
On 28/10/14 23:13, Clayton Kirkwood wrote: Explain this double speak(>: [pair for pair in values if key == pair[0]] A list comprehension is a specific form of a more general construction called a generator expression. Its specific in that it creates a list, hence the [] around it. The general shape of a generator expression is: for value in if and the comprehension version looks like [ for value in if ] The if condition part is optional in both cases. This is syntactically equivalent to aList = [] for value in iterator: if condition: aList.append(result expression) Let's look at a specific example using literals: [item for item in [1,2,3,4,5] if item > 3] The effect of which is the list [4,5] So it acts as a filter that extracts the items in the inner list that are greater than 3 (the if condition). In your case I had proposed using a list of tuples(pairs) like so: values = [(1,2),(2,3),(3,4),(4,5)] So the expression: [pair for pair in values if 2 in pair] will return [(1,2),(2,3)] and the expression [pair for pair in values if 2 == pair[0]] will return [(2,3)] - the only pair with 2 in the first position Now, coming back to your original question: [pair for pair in values if key == pair[0]] this returns the tuples that have the first element equal to key - whatever the variable key happens to be. PS. For bonus points you can research how generator expressions can encapsulate nested loops, but that's probably a step too far for just now! HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
On 28/10/14 23:13, Clayton Kirkwood wrote: line either a tuple or list. The comprehension seems to act as a subroutine or macro In a sense yes, that's true. You can find another explanation of list comprehensions and other related functions in my tutorial (see .sig) in the Functional Programming topic. It has several more examples. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had the some_list and pair the same name, presumably creating a temporary tuple and when the loop is done, the temporary replaces the original. Thanks Clayton !-Original Message- !From: Danny Yoo [mailto:d...@hashcollision.org] !Sent: Tuesday, October 28, 2014 4:23 PM !To: Clayton Kirkwood !Cc: Python Tutor Mailing List !Subject: Re: [Tutor] Would somebody kindly... ! !On Tue, Oct 28, 2014 at 4:13 PM, Clayton Kirkwood !wrote: !> Explain this double speak(>: !> !> [pair for pair in values if key == pair[0]] ! !Hi Clayton, ! !Here is a rewording of that expression to an approximately equivalent !statement: ! !### !some_list = [] !for pair in values: !if key == pair[0]: !some_list.append(pair) !### ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
On 29/10/14 01:02, Clayton Kirkwood wrote: Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had the some_list and pair the same name, presumably creating > a temporary tuple and when the loop is done, the temporary replaces the original. Nope. If you re-read my post you'll seee that my examples are just expressions, they don;t assign the result. Dannys code is actually the equivalent of: some_list = [pair for pair in values if key == pair[0]] some_list is an arbitrary name Danny used to represent the list that is built up by the comprehension. It doesn't actually have a name in the real comprehension. The name 'pair' in my code is used exactly as in Danny's 'for' loop below. !### !some_list = [] !for pair in values: !if key == pair[0]: !some_list.append(pair) !### There is no temporary tuples being created the list is built up inside the comprehension as the for part executes in the same way as in the expanded version. There is no magic here it's just a short-cut notation for the expanded loop (and it usually executes a bit faster). -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
> Explain this double speak(>: > [pair for pair in values if key == pair[0]] > I understand the ‘for pair in values’. I assume the first > ‘pair’ creates the namespace The namespace question depends on the version of Python. Python 2.x does not do any scoping. But in version 3.x, the variable pair will go away. So please tell us the version you're asking about. The list comprehension always creates a list, not a tuple. Pair is a tuple if and only if the corresponding element of values is a tuple. In fact there's no requirement that those elements of values have the same type. Of course if they're not of length 2, then pair is a lousy name. And if one of them is not subscriptable, then the if expression will blow up. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
On 28Oct2014 18:02, Clayton Kirkwood wrote: !> Explain this double speak(>: !> Ah, Alan sent an answer also, but this one answers the last tidbit. Alan had the some_list and pair the same name, presumably creating a temporary tuple and when the loop is done, the temporary replaces the original. Let me try a less wordy diagram. You will need to be displaying in a constant width font :-) [ pair for pair in values if key == pair[0] ] -- the expression that accrues in the resulting list -- the loop variable, taken from the loop source values ^^-- the loop source values ^^-- condition for including the expression in the resulting list So that first "pair" could be any expression, it is almost only coincidence that it is the same as the loop variable. It is the same in this case because this is the idiomatic way to select particular values form an existing list. If we'd wanted the new list to contain double the original values we'd write: [ pair*2 for pair in values if key == pair[0] ] Cheers, Cameron Simpson ERROR 155 - You can't do that. - Data General S200 Fortran error code list ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
!-Original Message- !From: Cameron Simpson [mailto:c...@zip.com.au] !Sent: Tuesday, October 28, 2014 6:31 PM !To: Clayton Kirkwood !Cc: 'Danny Yoo'; 'Python Tutor Mailing List' !Subject: Re: [Tutor] Would somebody kindly... ! !On 28Oct2014 18:02, Clayton Kirkwood wrote: !>!> Explain this double speak(>: !>!> !>Ah, Alan sent an answer also, but this one answers the last tidbit. !Alan had the some_list and pair the same name, presumably creating a !temporary tuple and when the loop is done, the temporary replaces the !original. ! !Let me try a less wordy diagram. You will need to be displaying in a !constant width font :-) ! ! [ pair for pair in values if key == pair[0] ] ! -- the expression that accrues in the resulting list ! -- the loop variable, taken from the loop source !values ! ^^-- the loop source values !^^-- condition for including !the !expression in the resulting !list ! !So that first "pair" could be any expression, it is almost only !coincidence that it is the same as the loop variable. It is the same in !this case because this is the idiomatic way to select particular values !form an existing list. ! !If we'd wanted the new list to contain double the original values we'd !write: ! ! [ pair*2 for pair in values if key == pair[0] ] Ok, I am somewhat confused again. In the original example up above, it appears that the pair list or tuple gets overridden. In this one right above, once again, the list gets overwritten again, but what is being doubled?, each part of the tuple? Clayton ! !Cheers, !Cameron Simpson ! !ERROR 155 - You can't do that. - Data General S200 Fortran error code !list ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Would somebody kindly...
!-Original Message- !From: Tutor [mailto:tutor-bounces+crk=godblessthe...@python.org] On !Behalf Of Dave Angel !Sent: Tuesday, October 28, 2014 6:34 PM !To: tutor@python.org !Subject: Re: [Tutor] Would somebody kindly... ! ! !> ! Explain this double speak(>: !> [pair for pair in values if key == pair[0]] ! !> I understand the ‘for pair in values’. I assume the first ‘pair’ !> creates the namespace ! !The namespace question depends on the version of Python. Python 2.x !does not do any scoping. ! !But in version 3.x, the variable pair will go away. ! !So please tell us the version you're asking about. I am using 3.4.1. ! !The list comprehension always creates a list, not a tuple. Pair is a !tuple if and only if the corresponding element of values is a tuple. In !fact there's no requirement that those elements of values have the same !type. Of course if they're not of length 2, then pair is a lousy name. !And if one of them is not subscriptable, then the if expression will !blow up. ! ! !-- !DaveA ! !___ !Tutor maillist - Tutor@python.org !To unsubscribe or change subscription options: !https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor