Karl Vogel put forth on 1/9/2011 6:04 PM: >>> On Sun, 09 Jan 2011 10:05:43 -0600, >>> Stan Hoeppner <s...@hardwarefreak.com> said: > > S> #! /bin/sh > S> for k in $(ls *.JPG); do convert $k -resize 1024 $k; done > > Someone was ragging on you to let the shell do the file expansion. I > like your way better because most scripting shells aren't smart enough > to realize that when there aren't any .JPG files, I don't want the > script to echo '*.JPG' as if that's actually useful.
This doesn't matter to me as I only use this script on a single temp directory after I dump the camera files into it. The camera, a Fujifilm FinePix A820 8.3MP, saves its files in all upper case. > S> I use the above script to batch re-size digital camera photos after I > S> dump them to my web server. It takes a very long time with lots of new > S> photos as the server is fairly old, even though it is a 2-way SMP, > S> because the script only runs one convert process at a time serially, > S> only taking advantage of one CPU. > > First things first: are you absolutely certain that running two parallel > jobs will exercise both CPUs? I've seen SMP systems that don't exactly > live up to truth-in-advertising. If you stuff two "convert" jobs in the > background and then run "top" (or the moral equivalent) do you SEE both > CPUs being worked? See my response to Bob. And see Bob's response to you. :) The issue you describe was resolved with a few patches many years ago, and only reared its ugly head on processors with SMT (HT) enabled. The kernel scheduler work lagged behind the hardware releases of IBM's SMT and Intel's HT. The chips were on the market a while before regular distro release cycles caught up. So early adopters of SMT chips saw the problem you describe. As Bob noted, in most situations, simply turning SMT off fixed the problem instantly. For those who don't know the acronyms, SMT stands for "Simultaneous Multi-threading", which is the textbook term for this technology. Intel gave their SMT implementation a catchy marketing name, HyperThreading, as they seem to do with every product, sadly. > Second: do you have "taskset" installed? If the work isn't being > divided up the way you like, you can bind a process to a desired core: > http://planet.admon.org/how-to-bind-a-certain-process-to-specified-core/ cpusets (see also cpumemsets) which is the kernel feature that tasksel manipulates, is overkill for managing process scheduling on a 2-way box, and wouldn't yield much, if any, benefit. In fact, if I were to attempt using it with my piddly workloads, I'd likely be far less efficient at manually scheduling tasks than the kernel. In fact, I can guarantee you of this. :) > And last: if you're not using something like LVM, can you do anything to > make sure you're not hitting the same disk? If all your new photos are > on the same drive, any CPU savings you get from parallel processing will > probably be erased by disk contention. Better yet, do you have enough > memory to do the processing on a RAM-backed filesystem? Apparently you've never used Imagemagick's convert utility, or any other image manipulation tools, or not on an older ~550MHz machine with tiny L2 cache (by today's standards). Image manipulation programs are always CPU bound, rarely, if ever, IO bound. I'd say "never" but I'm sure there is a rare corner case out there somewhere. It's odd isn't it, that I have pretty intimate knowledge of the things above, yet am handicapped WRT shell scripting? Nobody knows everything, and I'm sure glad lists such as debian-users exist to fill in the knowledge gaps. :) -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4d2ac7b7.3010...@hardwarefreak.com
