On Tue, 11 Jan 2011 15:58:45 -0600, Stan Hoeppner wrote: > Camaleón put forth on 1/11/2011 9:38 AM: > >> I supposed you wouldn't care much in getting a script to run faster >> with all the available core "occupied" if you had a modern (<4 years) >> cpu and plenty of speedy ram because the routine you wanted to run it >> should not take many time... unless you were going to process >> "thousand" of images :-) > > That's a bit ironic. You're suggesting the solution is to upgrade to a > new system with a faster processor and memory.
Why did you get that impression? No, I said I thought you were running a resource-scarce machine so in order to simulate your environment I made the tests under my VM... nothing more. > However, all the newer processors have 2, 4, 6, 8, or 12 cores. So > upgrading simply for single process throughput would waste all the > other cores, which was the exact situation I found myself in. But of course! I would not even think in upgrade the whole computer just to get one concrete task done a few more seconds faster. > The ironic part is that parallelizing the script to maximize performance > on my system will also do the same for the newer chips, but to an even > greater degree on those with 4, 6, 8, or 12 cores. Due to the fact that > convert doesn't eat 100% of a core's time during its run, and the idle > time in between one process finishing and xargs starting another, one > could probably run 16-18 parallel convert processes on a 12 core Magny > Cours with this script before run times stop decreasing. I think the script should also work very well with single-core cpus. > The script works. It cut my run time by over 50%. I'm happy. As I > said, this system's processing power is complete overkill 99% of the > time. It works beautifully with pretty much everything I've thrown at > it, for 8 years now. If I _really_ wanted to maximize the speed of this > photo resizing task I'd install Win32 ImageMagick on my 2GHz Athlon XP > workstation with dual channel memory nForce2 mobo, convert them on the > workstation, and copy them to the server. > > However, absolute maximum performance of this task was not, and is not > my goal. > My goal was to make use of the second CPU, which was sitting idle in > the > server, to speed up the task completion. That goal was accomplished. :) Yeah, and tests are there to demonstrate the gain. >>>> Running more processes than real cores seems fine, did you try it? >>> >>> Define "fine". >> >> Fine = system not hogging all resources. > > I had run 4 (2 core machine) and run time was a few seconds faster than > 2 processes, 3 seconds IIRC. Running 8 processes pushed the system into > swap and run time increased dramatically. Given that 4 processes only > have a few seconds faster than two, yet consumed twice as much memory, > the best overall number of processes to run on this system is two. Maybe the "best number of processes" is system-dependant (old processors could work better with a conservative value but newer ones can get some extra seconds with a higher one and without experiencing any significant penalty). Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pan.2011.01.12.09.56...@gmail.com
