Oh, and it's also worth mentioning that stack also takes "-j" to specify building more packages concurrently. It defaults to your processor count. It can be beneficial to use a higher value.
To pass the -j to the cabal builds, use --ghc-options "-j2" On Fri, Jul 8, 2016 at 12:31 PM, Michael Sloan <[email protected]> wrote: > On Fri, Jul 8, 2016 at 11:53 AM, Erik Rantapaa <[email protected]> > wrote: > >> Hi all, >> >> I've been building a large number of packages with a single `stack >> install` command, e.g. something like: >> >> stack install --resolver ... --keep-going $(cat all-packages) >> >> On a 8-core / 16 GB box I've noticed that the worker threads are not >> always kept busy. The overall %-idle time is at least 20%. >> > > >> >> So I'm wondering... >> >> - Is there anything I can do on the command line to improve CPU >> utilization? >> > > > You can definitely increase CPU utilization by specifying "-jN" options > for GHC (e.g. "-j2"). Unfortunately, in my experience it usually doesn't > help as much as I'd hope with overall build time. One issue is that it's > possible for it to interact poorly with package-level parallelism. For > example, if stack is building 5 packages with "-j5", suddenly there's > potentially 25 processes each trying to use 100% of a core. > > >> >> - Where can I find the code that does the job scheduling? I would like to >> modify how package builds are prioritized to improve throughput. >> > > Scheduling is done here: > https://github.com/commercialhaskell/stack/blob/master/src/Control/Concurrent/Execute.hs > > Converting the build tasks to actions for scheduling is here: > https://github.com/commercialhaskell/stack/blob/4304711d91f41a137931502ac4b1d00b3a4bc842/src/Stack/Build/Execute.hs#L573 > > > Thanks, >> Erik >> >> -- >> You received this message because you are subscribed to the Google Groups >> "haskell-stack" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/haskell-stack/329a3117-b2ff-411c-954d-156001b622d6%40googlegroups.com >> <https://groups.google.com/d/msgid/haskell-stack/329a3117-b2ff-411c-954d-156001b622d6%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "haskell-stack" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/haskell-stack/CAEYHaY6jQkuKUpa5E0pqNGgLOv47%2Bpqtb49wPat18%2Br1rBE9Jw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
