[issue10015] Creating a multiproccess.pool.ThreadPool from a child thread blows up.

2010-10-02 Thread Michael Olson 

New submission from Michael Olson :

Using Python 2.7 x32 on Windows XP

Attempting to create a multiprocessing.pool.ThreadPool
in a child thread created using threading.Thread, an
AttributeError is thrown. A ThreadPool created in the 
main thread can be passed to the child thread and used.


Exact text of exception
---
  File "D:\Dev\Python27\lib\multiprocessing\dummy\__init__.py", line 47, in star
t
self._parent._children[self] = None
AttributeError: 'Thread' object has no attribute '_children'


Demonstration Code
---
import unittest
from threading import Thread
from multiprocessing.pool import ThreadPool


def f(x):
return x*x


def create_and_run(cb, pool = None):
if not pool:
pool = ThreadPool(2)
r = pool.map_async(f, range(10))
cb(r.get())


class TestThreadPool(unittest.TestCase):
def setUp(self):
self.expected = [f(x) for x in range(10)]

def callback(self, data):
self.data = data

def test_creating_pool_in_mainthread(self):
"""Test multiprocessing.pool.ThreadPool from main thread"""
self.data = None
create_and_run(self.callback)
self.assertEqual(self.data, self.expected)

def test_creating_pool_in_subthread(self):
"""Test multiprocessing.pool.ThreadPool from a child thread."""
self.data = None
t = Thread(target=create_and_run, args=[self.callback])
t.start()
t.join()
self.assertEqual(self.data, self.expected)

def test_creating_pool_in_subthread_workaround(self):
"""Test running a ThreadPool created in main thread, used in child."""
self.data = None
pool = ThreadPool(2)
t = Thread(target=create_and_run, args=[self.callback, pool])
t.start()
t.join()
self.assertEqual(self.data, self.expected)


if __name__ =='__main__':
suite = unittest.TestLoader().loadTestsFromTestCase(TestThreadPool)
unittest.TextTestRunner(verbosity=2).run(suite)

--
components: Library (Lib)
files: potential_issue_demo.py
messages: 117875
nosy: Michael.Olson
priority: normal
severity: normal
status: open
title: Creating a multiproccess.pool.ThreadPool from a child thread blows up.
type: crash
versions: Python 2.7
Added file: http://bugs.python.org/file19105/potential_issue_demo.py

___
Python tracker 
<http://bugs.python.org/issue10015>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10128] multiprocessing.Pool throws exception with __main__.py

2010-10-16 Thread Michael Olson 

New submission from Michael Olson :

In an application with an entry point of __main__.py, multiprocessing.Pool 
throws the following:

Traceback (most recent call last):
  File "", line 1, in 
  File "D:\Dev\Python27\lib\multiprocessing\forking.py", line 346, in main
prepare(preparation_data)
  File "D:\Dev\Python27\lib\multiprocessing\forking.py", line 454, in prepare
assert main_name not in sys.modules, main_name
AssertionError: __main__

These messages repeat as long as the application is running.

Demonstration Code, must be in file named __main__.py:

import multiprocessing
import time

if __name__ == '__main__':
pool = multiprocessing.Pool()
time.sleep(2)


--
components: Library (Lib)
messages: 118905
nosy: Michael.Olson
priority: normal
severity: normal
status: open
title: multiprocessing.Pool throws exception with __main__.py
type: behavior
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue10128>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10128] multiprocessing.Pool throws exception with __main__.py

2010-10-17 Thread Michael Olson 

Michael Olson  added the comment:

I wrapped the offending assertion in a if main_name != '__main__'. I considered 
not checking the module_name against built-in modules but that seemed likely to 
be the sort of thing being guarded against, so I left it at an exception for 
__main__.

Ran a few trivial testing using Pool and Process and it seems to work fine.

--

___
Python tracker 
<http://bugs.python.org/issue10128>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10128] multiprocessing.Pool throws exception with __main__.py

2010-10-19 Thread Michael Olson 

Michael Olson  added the comment:

Sorry about that, yes, this is on Windows XP and 7, 32 bit.

And with the if statement it seems to work fine.

v/r
-- Michael Olson

--

___
Python tracker 
<http://bugs.python.org/issue10128>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10128] multiprocessing.Pool throws exception with __main__.py

2010-10-20 Thread Michael Olson 

Michael Olson  added the comment:

Ummm, I think I've been unclear on where I was making changes, I changed 
lib\multiprocessing\forking.py to fix the issue.

Patch attached.

--
keywords: +patch
resolution: invalid -> 
status: closed -> open
Added file: http://bugs.python.org/file19296/issue10128.patch

___
Python tracker 
<http://bugs.python.org/issue10128>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10128] multiprocessing.Pool throws exception with __main__.py

2010-10-20 Thread Michael Olson 

Michael Olson  added the comment:

As a note, I didn't attach a patch at first because I was fairly sure I was 
kludging it into submission, but at least this makes it clear as to what I did.

v/r
-- Michael Olson

--

___
Python tracker 
<http://bugs.python.org/issue10128>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com