Starting an OpenMP parallel section is extremely slow on a hyper-threaded Nehalem

[Edwin Bennink]

Dear gcc list,


I noticed that starting an OpenMP parallel section takes a significant 
amount of time on Nehalem cpu's with hyper-threading enabled.

The differences with HTT turned on and off are really huge:

- HTT disabled: about 100.000 parallel sections per second
- HTT enabled: about 15 parallel sections per second

Is this a known problem? It has apparently something to do with setting 
the cpu affinity; when I set the GOMP_CPU_AFFINITY environment variable 
to "0-7", then it is almost as fast as with HTT disabled...



This is the code I used to test it. Simply compile it with -fopenmp. I 
used 100.000 iterations instead of 100 to time it with HTT disabled.




int main () {
   int i;

   for (i = 0; i < 100; i++) {
#pragma omp parallel
   {
   }
   }
}




System specs:
OS: Ubuntu 9.10, amd64 (2.6.31-19-generic)
gcc: version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)
cpu: Intel(R) Core(TM) i7 CPU 920  @ 2.67GHz


Cheers,
Edwin

Re: Starting an OpenMP parallel section is extremely slow on a hyper-threaded Nehalem

[Edwin Bennink]

Thanks Tim, I thought that the gcc list was the most appropriate one 
regarding the gomp implementation, but I'll post this question on the 
gcc-help list.


By the way, Ubuntu 9.10 is the latest version (dd Oct. 2009). HTT works 
fine for daily use, but massive parallel applications show some odd 
behaviour:
Depending on the structure of the algorithm some pieces of code run 
significantly faster (about 10%) with HTT enabled, while other pieces of 
code run slower (some more than 50%). This slowdown happens due to 
parallel sections inside loops...


Edwin


Tim Prince wrote:

On 2/11/2010 2:00 AM, Edwin Bennink wrote:

Dear gcc list,


I noticed that starting an OpenMP parallel section takes a 
significant amount of time on Nehalem cpu's with hyper-threading 
enabled.


If you think a question might be related to gcc, but don't know which 
forum to use, gcc-help is more appropriate.  As your question is 
whether there is a way to avoid anomalous behaviors when an old Ubuntu 
is run on a CPU released after that version of Ubuntu, an Ubuntu forum 
might be more appropriate.  A usual way is to shut off HyperThreading 
in the BIOS when running on a distro which has trouble with it.  I do 
find your observation interesting.
As far as I know, the oldest distro which works well on Core I7 is 
RHEL5.2 x86_64, which I run, with updated gcc and binutils, and HT 
disabled, as I never run applications which could benefit from HT.