@Nataniel this is what I am suggesting as well. No cacheing - just storing
the `fn` on each worker, rather than pickling it for each item in our
iterable.
As long as we store the `fn` post-fork on the worker process (perhaps as
global), subsequent calls to Pool.map shouldn't be effected (referenci
Would this change the other pool method behavior in some way if the user,
for whatever reason, mixed techniques?
imap_unordered will only block when nexting the generator. If the user
mingles nexting that generator with, say, apply_async, could the change
you're proposing have some side-effect?
O
On Sun, Oct 14, 2018 at 12:15 PM Jeff Allen wrote:
>
>
> On 10/10/2018 00:06, Steven D'Aprano wrote:
>
> On Tue, Oct 09, 2018 at 09:37:48AM -0700, Jeff Hardy wrote:
>
> ...
>
> From an alternative implementation point of view, CPython's behaviour
> *is* the spec. Practicality beats purity and all
Is your concern something like the following?
with Pool(8) as p:
gen = p.imap_unordered(func, ls)
first_elem = next(gen)
p.apply_async(long_func, x)
remaining_elems = [elem for elem in gen]
...here, if we store "func" on each worker Process as a global, and execute
this pattern ab
