total newb here
I've bought a few books on the topic, but I'm a total newb to programming in general. I've got about 4 years PHP / MySQL experience, but none with "actual programming". To be honest, I don't even really know what Python is used for. I'm thought I would start here and see if anyone knew of good places to start for someone as green as I. And now I'm off to google. Thanks in advance. Greg -- http://mail.python.org/mailman/listinfo/python-list
using print() with multiprocessing and pythonw
I launch my program with pythonw and begin it with the code below so that all
my print()'s go to the log file specified.
if sys.executable.find('pythonw') >=0:
# Redirect all console output to file.
sys.stdout = open("pythonw - stdout stderr.log",'w')
sys.stderr = sys.stdout
During the course of my program, I call multiprocessing.Process() and launch a
function several times. That function has print()'s inside (which are from
warnings being printed by python). This printing causes the multiprocess to
crash. How can I fix my code so that the print()'s are supressed. I would hate
to do a warnings.filterwarnings('ignore') because when I unit test those
functions, the warnings dont appear.
Thanks in advance,
Isaac
--
https://mail.python.org/mailman/listinfo/python-list
Re: using print() with multiprocessing and pythonw
Thanks for the reply Bill. The problem is the text i am getting is from a python warning message, not one of my own print() function calls. -- https://mail.python.org/mailman/listinfo/python-list
Python 3.2 | WIndows 7 -- Multiprocessing and files not closing
I have a function that looks like the following: #- filename = 'c:\testfile.h5' f = open(filename,'r') data = f.read() q = multiprocessing.Queue() p = multiprocess.Process(target=myFunction,args=(data,q)) p.start() result = q.get() p.join() q.close() f.close() os.remove(filename) #- When I run this code, I get an error on the last line when I try to remove the file. It tells me that someone has access to the file. When I remove the queue and multiprocessing stuff, the function works fine. What is going on here? Thanks in advance, Isaac -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing
Sorry, I am just providing pseudo code since I the code i have is quite large. As I mentioned, the code works fine when I remove the multirpcessing stuff so the filename is not the issue (though you are right in your correction). Someone with the same problem posted a smaller, more complete example here: http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib None of the solutions posted work. On Thursday, October 10, 2013 12:38:19 PM UTC-4, Piet van Oostrum wrote: > Isaac Gerg writes: > > > > > I have a function that looks like the following: > > > > That doesn't look like a function > > > > > > > > #- > > > filename = 'c:\testfile.h5' > > > > Your filename is most probably wrong. It should be something like: > > > > filename = 'c:/testfile.h5' > > filename = 'c:\\testfile.h5' > > filename = r'c:\testfile.h5' > > -- > > Piet van Oostrum > > WWW: http://pietvanoostrum.com/ > > PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing
On Thu, Oct 10, 2013 at 2:41 PM, Ned Batchelder wrote: > On 10/10/13 12:44 PM, Isaac Gerg wrote: > >> Sorry, I am just providing pseudo code since I the code i have is quite >> large. >> >> As I mentioned, the code works fine when I remove the multirpcessing >> stuff so the filename is not the issue (though you are right in your >> correction). >> >> Someone with the same problem posted a smaller, more complete example >> here: >> >> http://stackoverflow.com/**questions/948119/preventing-** >> file-handle-inheritance-in-**multiprocessing-lib<http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib> >> >> None of the solutions posted work. >> > > (BTW: it's better form to reply beneath the original text, not above it.) > > None of the solutions try the obvious thing of closing the file before > spawning more processes. Would that work for you? A "with" statement is a > convenient way to do this: > > with open(filename,'r') as f: > data = f.read() > > The file is closed automatically when the with statement ends. > > --Ned. > > >> On Thursday, October 10, 2013 12:38:19 PM UTC-4, Piet van Oostrum wrote: >> >>> Isaac Gerg writes: >>> >>> >>> >>> I have a function that looks like the following: >>>> >>> >>> >>> That doesn't look like a function >>> >>> >>> >>> #-** >>>> filename = 'c:\testfile.h5' >>>> >>> >>> >>> Your filename is most probably wrong. It should be something like: >>> >>> >>> >>> filename = 'c:/testfile.h5' >>> >>> filename = 'c:\\testfile.h5' >>> >>> filename = r'c:\testfile.h5' >>> >>> -- >>> >>> Piet van Oostrum >>> >>> WWW: http://pietvanoostrum.com/ >>> >>> PGP key: [8DAE142BE17999C4] >>> >> > I will try what you suggest and see if it works. Additionally, is there a place on the web to view this conversation and reply? Currently, I am only able to access this list through email. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing
On Thu, Oct 10, 2013 at 2:49 PM, Isaac Gerg wrote: > > > > On Thu, Oct 10, 2013 at 2:41 PM, Ned Batchelder wrote: > >> On 10/10/13 12:44 PM, Isaac Gerg wrote: >> >>> Sorry, I am just providing pseudo code since I the code i have is quite >>> large. >>> >>> As I mentioned, the code works fine when I remove the multirpcessing >>> stuff so the filename is not the issue (though you are right in your >>> correction). >>> >>> Someone with the same problem posted a smaller, more complete example >>> here: >>> >>> http://stackoverflow.com/**questions/948119/preventing-** >>> file-handle-inheritance-in-**multiprocessing-lib<http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib> >>> >>> None of the solutions posted work. >>> >> >> (BTW: it's better form to reply beneath the original text, not above it.) >> >> None of the solutions try the obvious thing of closing the file before >> spawning more processes. Would that work for you? A "with" statement is a >> convenient way to do this: >> >> with open(filename,'r') as f: >> data = f.read() >> >> The file is closed automatically when the with statement ends. >> >> --Ned. >> >> >>> On Thursday, October 10, 2013 12:38:19 PM UTC-4, Piet van Oostrum wrote: >>> >>>> Isaac Gerg writes: >>>> >>>> >>>> >>>> I have a function that looks like the following: >>>>> >>>> >>>> >>>> That doesn't look like a function >>>> >>>> >>>> >>>> #-** >>>>> filename = 'c:\testfile.h5' >>>>> >>>> >>>> >>>> Your filename is most probably wrong. It should be something like: >>>> >>>> >>>> >>>> filename = 'c:/testfile.h5' >>>> >>>> filename = 'c:\\testfile.h5' >>>> >>>> filename = r'c:\testfile.h5' >>>> >>>> -- >>>> >>>> Piet van Oostrum >>>> >>>> WWW: http://pietvanoostrum.com/ >>>> >>>> PGP key: [8DAE142BE17999C4] >>>> >>> >> > I will try what you suggest and see if it works. > > Additionally, is there a place on the web to view this conversation and > reply? Currently, I am only able to access this list through email. > Ned, I am unable to try what you suggest. The multiprocess.Process call is within a class but its target is a static method outside of the class thus no pickling. I cannot close the file and then reopen after the multiprocess.Process call because other threads may be reading from the file during that time. Is there a way in Python 3.2 to prevent the multiprocess.Process from inheriting the file descriptors from the parent process OR, is there a way to ensure that the multiprocess is completely closed and garbaged collected by the time I want to use os.remove()? Isaac -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.2 | WIndows 7 -- Multiprocessing and files not closing
Hi Piet, Here is a real code example: http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib As I said before, I had provide pseudocode. I cannot close the file after reading because it is part of a class and other threads may be calling member functions which read from the file. Isaac -- https://mail.python.org/mailman/listinfo/python-list
