[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Changes by Terry J. Reedy : -- assignee: -> d...@python components: +Documentation -Library (Lib) nosy: +d...@python resolution: -> fixed status: open -> closed versions: +Python 2.7 -Python 2.5, Python 2.6 ___ Python tracker

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Antoine Pitrou added the comment: > This problem has come up before. It was a bug introduced I think when > the subprocess module was first incorporated into Python. I don't > recall if the default was changed in 2.7 or 3.x. According to the docs it is still unbuffered by default. I've added

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Skip Montanaro added the comment: This problem has come up before. It was a bug introduced I think when the subprocess module was first incorporated into Python. I don't recall if the default was changed in 2.7 or 3.x. Can you demonstrate the problem with Python 2.7 or 3.1? If not, I'd recom

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Antoine Pitrou added the comment: > I think the default of 0, meaning unbuffered, is a poor choice. So do I, but I'm not sure whether it's acceptable to change behaviour in a potentially incompatible way. > Barring that semantic change, please add a note to the doc: "Most > users will want to

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Hugh Secker-Walker added the comment: OK. Using bufsize=4096 or bufsize=-1 eliminates the CPU-hogging problem on Mac OS X. Thank you. I have to say that the Popen() doc could be better. It says: "bufsize, if given, has the same meaning as the corresponding argument to the built-in open()

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Antoine Pitrou added the comment: Have you tried specifying a value for the bufsize argument to Popen? Either 1 for line buffering or e.g. 4096 for a decent size block buffering. -- nosy: +gregory.p.smith, pitrou ___ Python tracker

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Changes by Hugh Secker-Walker : Added file: http://bugs.python.org/file17519/pull1 ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

Changes by Hugh Secker-Walker : Added file: http://bugs.python.org/file17518/pull0 ___ Python tracker ___ ___ Python-bugs-list mailing list Uns

[issue8873] Popen uses 333 times as much CPU as a shell pipe on Mac OS X

New submission from Hugh Secker-Walker : Popen seems to be *very* costly, CPU wise, on Mac OS X. I need to read and process bytes from the stdout of another process that outputs bytes in a rate-limited fashion, e.g. 8 bytes per second. If I use a shell pipeline and a simple Python script