On Fri, Aug 5, 2016, at 01:33 PM, Gregory Szorc wrote: > > The overhead of serialization/deserialization + process startup has a > strong chance of canceling out any perf wins with Python. > Despite the GIL, Python can get nice performance wins when using > multiple threads for I/O bound tasks (including network operations). > If all it is doing is writing files, try plugging in a > concurrent.futures.ThreadPoolExecutor to do the file writing. Search > the tree for "ThreadPoolExecutor" to find example usage.
This is actually slower on my machine. Existing code: luser@eye7:/build/mozilla-central$ for x in dom/webidl/*.webidl; do echo >> $x; done; time ./mach build dom/bindings/codegen.pp 0:00.12 /usr/bin/make -C /build/debug-mozilla-central -j8 -s backend 0:00.20 /usr/bin/make -C dom/bindings -j8 -s codegen.pp 0:14.64 Overall system resources - Wall time: 14s; CPU: 15%; Read bytes: 0; Write bytes: 9306112; Read time: 0; Write time: 1544 0:14.65 Your build was successful! To view resource usage of the build, run |mach resource-usage|. real 0m14.789s user 0m14.664s sys 0m0.406s With ThreadPoolExecutor (https://diff.pastebin.mozilla.org/8889858): luser@eye7:/build/mozilla-central$ for x in dom/webidl/*.webidl; do echo >> $x; done; time ./mach build dom/bindings/codegen.pp 0:00.12 /usr/bin/make -C /build/debug-mozilla-central -j8 -s backend 0:00.19 /usr/bin/make -C dom/bindings -j8 -s codegen.pp 0:23.19 Overall system resources - Wall time: 23s; CPU: 32%; Read bytes: 16384; Write bytes: 23060480; Read time: 0; Write time: 3076 0:23.20 Your build was successful! To view resource usage of the build, run |mach resource-usage|. real 0m23.335s user 0m23.067s sys 0m3.477s -Ted
_______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
