even though I can't supply the machine, I thought the techniques could be
useful. most computers are multicore nowadays, so there is a capability for
parallel builds. to find out how much memory per thread or core you have,
divide the available RAM by the number of cores.
the individual compiler instance will use up so much RAM. you probably don't
want to let it get below 4-8GB/compiler instance.
you don't have to go out and buy a new box to do this. you can use an existing
box and do parallel builds and save time, taking advantage of the multicore cpu.
Just a thought.
the basic idea I had a while back, and was "tuned" by adobe in their
performance video.
http://tv.adobe.com/watch/maximizing-cs55-performance-with-intel-xeon-processors/how-to-configure-a-highperformance-workstation-for-production-premium-cs55/
Jim Michaels
>________________________________
> From: NightStrike <[email protected]>
>To: "[email protected]"
><[email protected]>
>Sent: Saturday, January 5, 2013 7:21 AM
>Subject: Re: [Mingw-w64-public] build boxes
>
>On Sat, Jan 5, 2013 at 12:08 AM, Jim Michaels <[email protected]> wrote:
>> http://exxactcorp.com/index.php/solution/solu_list/53
>>
>> http://exxactcorp.com/index.php/solution/solu_detail/42
>> this one I don't know if it runs windows or not. but it is an HPC server
>> with up to 40 cores and 80 threads, 1400W redundant PSU, up to 512GB RAM (so
>> you can have up to 6.4GB/thread).
>
>Thanks for the link. Are you offering to buy it and host it?
>
>> if you are doing a lot of parallel builds, this might be a solution. or
>> maybe you just want to stick to sequential builds instead of doing them in
>> parallel? note that doing this in parallel, it would peg the disk hard, and
>> there is probably a performance limit on the amount of things you could do
>> in parallel before hitting that barrier based on your OS and filesystem
>> chosen.
>>
>> perhaps the introduction of a compiler that compiles multithreadedly would
>> be an idea? I don't know how that would work, or how you would divvy up the
>> work. I don't know whether to
>>
>> what I was thinking, was that at least, the individual builds could be
>> parallel-compiled: if you have a master makefile or script which batches
>> these things off, simply run them as background jobs by appending a & to the
>> command. the trick is synchronizing. you have to find some way of waiting
>> for the jobs to become *done* before you go on to the next batch. I will
>> leave this to you.
>> one way to do this is by leaving a sentinel file named something like
>> build-x.done
>> in your main thread/master script, you would have to wait in a loop for this
>> file to exist, and then delete it. if you have a group of processes, then
>> you have to check for the existence of the group of files before exiting the
>> loop like a great big AND.
>> this might get you thinking in parallel.
>>
>> these are really basic ideas, but they should work. if you need to kill the
>> jobs, it would be a task in itself, I would suggest you write a batch file
>> which greps ps for the different compiler batch processes (make, the name of
>> any master batch file, etc., and then
>> leaving the .done flies is not an issue. deleting them would be if the
>> processes are hanging around and any dependent processes are running.
>>
>> at a smaller scale, the file compiles could be parallel-compiled. there are
>> usually plenty of those.
>>
>> -------------
>> Jim Michaels
>> [email protected]
>> [email protected]
>> http://RenewalComputerServices.com
>> http://JesusnJim.com (my personal site, has software)
>> ---
>> IEC Units: Computer RAM & SSD measurements, microsoft disk size measurements
>> (note: they will say GB or MB or KB or TB when it is IEC Units!):
>> [KiB] [MiB] [GiB] [TiB]
>> [2^10B=1,024^1B=1KiB]
>> [2^20B=1,024^2B=1,048,576B=1MiB]
>> [2^30B=1,024^3B=1,073,741,824B=1GiB]
>> [2^40B=1,024^4B=1,099,511,627,776B=1TiB]
>> [2^50B=1,024^5B=1,125,899,906,842,624B=1PiB]
>> SI Units: Hard disk industry disk size measurements:
>> [KB] [MB] [GB] [TB]
>> [10^3B=1,000B=1KB]
>> [10^6B=1,000,000B=1MB]
>> [10^9B=1,000,000,000B=1GB]
>> [10^12B=1,000,000,000,000B=1TB]
>> [10^15B=1,000,000,000,000,000B=1PB]
>>
>>
>> ------------------------------------------------------------------------------
>> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>> MVPs and experts. SALE $99.99 this month only -- learn more at:
>> http://p.sf.net/sfu/learnmore_122912
>> _______________________________________________
>> Mingw-w64-public mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>>
>
>------------------------------------------------------------------------------
>Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
>MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
>with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
>MVPs and experts. SALE $99.99 this month only -- learn more at:
>http://p.sf.net/sfu/learnmore_122912
>_______________________________________________
>Mingw-w64-public mailing list
>[email protected]
>https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
>
>
>
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122412
_______________________________________________
Mingw-w64-public mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
