On Fri, Aug 5, 2016 at 11:21 AM, Ted Mielczarek <t...@mielczarek.org> wrote:
> On Fri, Aug 5, 2016, at 01:33 PM, Gregory Szorc wrote: > > > The overhead of serialization/deserialization + process startup has a > strong chance of canceling out any perf wins with Python. > Despite the GIL, Python can get nice performance wins when using multiple > threads for I/O bound tasks (including network operations). If all it is > doing is writing files, try plugging in a > concurrent.futures.ThreadPoolExecutor > to do the file writing. Search the tree for "ThreadPoolExecutor" to find > example usage. > > > This is actually slower on my machine. > > Existing code: > luser@eye7:/build/mozilla-central$ for x in dom/webidl/*.webidl; do echo > >> $x; done; time ./mach build dom/bindings/codegen.pp > 0:00.12 /usr/bin/make -C /build/debug-mozilla-central -j8 -s backend > 0:00.20 /usr/bin/make -C dom/bindings -j8 -s codegen.pp > 0:14.64 Overall system resources - Wall time: 14s; CPU: 15%; Read bytes: > 0; Write bytes: 9306112; Read time: 0; Write time: 1544 > 0:14.65 Your build was successful! > To view resource usage of the build, run |mach resource-usage|. > > real 0m14.789s > user 0m14.664s > sys 0m0.406s > > With ThreadPoolExecutor (https://diff.pastebin.mozilla.org/8889858): > luser@eye7:/build/mozilla-central$ for x in dom/webidl/*.webidl; do echo > >> $x; done; time ./mach build dom/bindings/codegen.pp > 0:00.12 /usr/bin/make -C /build/debug-mozilla-central -j8 -s backend > 0:00.19 /usr/bin/make -C dom/bindings -j8 -s codegen.pp > 0:23.19 Overall system resources - Wall time: 23s; CPU: 32%; Read bytes: > 16384; Write bytes: 23060480; Read time: 0; Write time: 3076 > 0:23.20 Your build was successful! > To view resource usage of the build, run |mach resource-usage|. > > real 0m23.335s > user 0m23.067s > sys 0m3.477s > Oof. I really don't like the GIL :/
_______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
