Re: [Rd] [R] Building with MKL on Ubuntu
On Tue, Nov 4, 2008 at 7:34 PM, <[EMAIL PROTECTED]> wrote: > I can see a couple of problems: > > 1) No "-lpthread" in --with-blas > 2) No "-L" prefix in the library path in --with-lapack > > In addition, I don't think you need to add -lmkl to --with-lapack, > although that is probably harmless. > > Martyn > > Quoting Prof Brian Ripley <[EMAIL PROTECTED]>: > > > Look in config.log to see what's wrong. (E.g. is > > /opt/intel/mkl/10.0.2.018/lib/em64t in the ld.so cache?) > > > > And note the warnings in the manual about using --with-lapack: it is most > > definitely not recommended. > > > > R-devel would be a better place to ask questions about this. > > > Thanks Brian and Martyn, I've tried it again with two sets of configure options: ./configure --with-blas='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl -lguide -lpthread' --with-lapack='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl_lapack' --enable-R-shlib ./configure --with-blas='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl -lguide -lpthread' --enable-R-shlib In both cases, the result is the same: R builds and uses its own BLAS. I didn't find anything that looked like evidence that /opt/intel/mkl/ 10.0.2.018/lib/em64t was getting into the ld.so cache, ie I didn't find 'ld.so' and 'opt/intel...' close together anywhere. Summaries of the BLAS- or MKL-related bits in config.log follow, with the first set of options: configure:36563: checking for dgemm_ in -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread configure:36594: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread -lgfortran -lm -ldl -lm >&5 conftest.c: In function 'main': conftest.c:187: warning: implicit declaration of function 'dgemm_' configure:36600: $? = 0 configure:36616: result: yes configure:37408: checking whether double complex BLAS can be used configure:37481: result: no ... BLAS_LIBS0='' BLAS_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRblas' BLAS_SHLIB_FALSE='#' BLAS_SHLIB_TRUE='' ... JAVA_LD_LIBRARY_PATH='$(JAVA_HOME)/lib/amd64/server:$(JAVA_HOME)/lib/amd64:$(JAVA_HOME)/../lib/amd64:/usr/local/lib64::/usr/lib64:/lib64:/usr/local/lib64:/usr/lib:/usr/local/lib:/lib:/opt/intel/fce/10.1.018/lib:/opt/intel/ipp/ 5.3.4.080/em64t/sharedlib:/opt/intel/cce/10.1.018/lib:/opt/intel/mkl/10.0.2.018/lib/em64t:/usr/java/packages/lib/amd64:/lib:/usr/lib ' JAVA_LIBS0='-L$(JAVA_HOME)/lib/amd64/server -L$(JAVA_HOME)/lib/amd64 -L$(JAVA_HOME)/../lib/amd64 -L/usr/local/lib64 -L -L/usr/lib64 -L/lib64 -L/usr/local/lib64 -L/usr/lib -L/usr/local/lib -L/lib -L/opt/intel/fce/10.1.018/lib -L/opt/intel/ipp/5.3.4.080/em64t/sharedlib-L/opt/intel/cce/10.1.018/lib -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -L/usr/java/packages/lib/amd64 -L/lib -L/usr/lib -ljvm' LAPACK_LDFLAGS='' LAPACK_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRlapack' and with the second set: configure:36563: checking for dgemm_ in -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread configure:36594: gcc -std=gnu99 -o conftest -g -O2 -fpic -I/usr/local/include -L/usr/local/lib64 conftest.c -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -lmkl -lguide -lpthread -lgfortran -lm -ldl -lm >&5 conftest.c: In function 'main': conftest.c:187: warning: implicit declaration of function 'dgemm_' configure:36600: $? = 0 configure:36616: result: yes configure:37408: checking whether double complex BLAS can be used configure:37481: result: no ... BLAS_LIBS0='' BLAS_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRblas' BLAS_SHLIB_FALSE='#' BLAS_SHLIB_TRUE='' ... JAVA_LD_LIBRARY_PATH='$(JAVA_HOME)/lib/amd64/server:$(JAVA_HOME)/lib/amd64:$(JAVA_HOME)/../lib/amd64:/usr/local/lib64::/usr/lib64:/lib64:/usr/local/lib64:/usr/lib:/usr/local/lib:/lib:/opt/intel/fce/10.1.018/lib:/opt/intel/ipp/ 5.3.4.080/em64t/sharedlib:/opt/intel/cce/10.1.018/lib:/opt/intel/mkl/10.0.2.018/lib/em64t:/usr/java/packages/lib/amd64:/lib:/usr/lib ' JAVA_LIBS0='-L$(JAVA_HOME)/lib/amd64/server -L$(JAVA_HOME)/lib/amd64 -L$(JAVA_HOME)/../lib/amd64 -L/usr/local/lib64 -L -L/usr/lib64 -L/lib64 -L/usr/local/lib64 -L/usr/lib -L/usr/local/lib -L/lib -L/opt/intel/fce/10.1.018/lib -L/opt/intel/ipp/5.3.4.080/em64t/sharedlib-L/opt/intel/cce/10.1.018/lib -L/opt/intel/mkl/ 10.0.2.018/lib/em64t -L/usr/java/packages/lib/amd64 -L/lib -L/usr/lib -ljvm' LAPACK_LDFLAGS='' LAPACK_LIBS='-L$(R_HOME)/lib$(R_ARCH) -lRlapack' Do either of these give us clues as to what's wrong? Thanks, Anand [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Building with MKL on Ubuntu
On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer <[EMAIL PROTECTED]> wrote:
> It looks like the em64t version of MKL fails the test for the accuracy
> of zdotu ("checking whether double complex BLAS can be used") and is
> therefore dropped in favour of R's built-in BLAS. I have just tested
> this on Fedora and get the same result.
>
> The 32-bit MKL does work for me.
>
> Martyn
>
Many thanks, Martyn. The 64-bit index space is important to me, will I lose
it if I link against the 32-bit MKL? Also, should I file a bug report with
Intel?
Anand
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Building with MKL on Ubuntu
Thanks for your help, everyone. I was using gcc previously, but building
with the Intel compilers causes its own problems:
/opt/intel/cce/10.1.018/lib/libguide.so: undefined reference to
`pthread_atfork'
make[3]: *** [R.bin] Error 1
make[3]: Leaving directory `/working_copies/R/src/main'
make[2]: *** [R] Error 2
make[2]: Leaving directory `/working_copies/R/src/main'
make[1]: *** [R] Error 1
make[1]: Leaving directory `/working_copies/R/src'
make: *** [R] Error 1
If your experience with MKL has been negative, I've definitely had enough;
I'll try my luck with gcc and GotoBLAS.
Cheers,
Anand
On Wed, Nov 5, 2008 at 9:02 PM, <[EMAIL PROTECTED]> wrote:
> Quoting Peter Dalgaard <[EMAIL PROTECTED]>:
>
> > Anand Patil wrote:
> > > On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer <[EMAIL PROTECTED]>
> wrote:
> > >
> > >> It looks like the em64t version of MKL fails the test for the accuracy
> > >> of zdotu ("checking whether double complex BLAS can be used") and is
> > >> therefore dropped in favour of R's built-in BLAS. I have just tested
> > >> this on Fedora and get the same result.
> > >>
> > >> The 32-bit MKL does work for me.
> > >>
> > >> Martyn
> > >>
> > >
> > > Many thanks, Martyn. The 64-bit index space is important to me, will I
> lose
> > > it if I link against the 32-bit MKL? Also, should I file a bug report
> with
> > > Intel?
> > > Anand
> >
> > There's also the option of "breaking the thermometer". You might examine
> > that check and decide whether the loss of accuracy is enough for you to
> > worry about and if not, take out the test from configure.
> >
> > Apparently, this check was put in place in R-2.2.0
> >
> > o Any external BLAS found is now tested to see if the complex
> > routine zdotu works correctly: this provides a compatibility
> > test of compiler return conventions.
> >
> > which suggests that the expected failure is catastrophic, and looking at
> > the code, there's a fuzz of 1e-10 which would seem to be about 1e5 times
> > larger than required.
>
> My apologies. I skimmed the code of the test program, but as Brian says
> it is not a question of accuracy. The test program does in fact segfault,
> although you won't see this in your config.log.
>
> Anyway, Anand, I would just carry on with libRblas.
>
> Martyn
>
>
>
> ---
> This message and its attachments are strictly confiden...{{dropped:12}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Building with MKL on Ubuntu
Scratch that- finally got it working! The steps were:
- Change configure to 'break the thermometer' as suggested:
===
--- configure (revision 46843)
+++ configure (working copy)
@@ -37473,6 +37473,8 @@
fi
+r_cv_zdotu_is_usable=yes
+
rm -rf conftest conftest.* conftestf.* core
if test -n "${r_cv_zdotu_is_usable}"; then
{ echo "$as_me:$LINENO: result: yes" >&5
- Set environment variables:
CFLAGS=-pthread -O3
FC=gfortran -pthread
FFLAGS=-pthread
CXXFLAGS=-O3 -pthread
FCLAGS=-pthread
LDFLAGS=-lpthread
- Configure command:
./configure --with-blas='-L/opt/intel/mkl/10.0.2.018/lib/em64t -lmkl -lguide
-lpthread' --enable-R-shlib
I owe you all a beer!
Anand
On Wed, Nov 5, 2008 at 9:10 PM, Anand Patil <[EMAIL PROTECTED]
> wrote:
> Thanks for your help, everyone. I was using gcc previously, but building
> with the Intel compilers causes its own problems:
> /opt/intel/cce/10.1.018/lib/libguide.so: undefined reference to
> `pthread_atfork'
> make[3]: *** [R.bin] Error 1
> make[3]: Leaving directory `/working_copies/R/src/main'
> make[2]: *** [R] Error 2
> make[2]: Leaving directory `/working_copies/R/src/main'
> make[1]: *** [R] Error 1
> make[1]: Leaving directory `/working_copies/R/src'
> make: *** [R] Error 1
>
> If your experience with MKL has been negative, I've definitely had enough;
> I'll try my luck with gcc and GotoBLAS.
>
> Cheers,
> Anand
>
>
> On Wed, Nov 5, 2008 at 9:02 PM, <[EMAIL PROTECTED]> wrote:
>
>> Quoting Peter Dalgaard <[EMAIL PROTECTED]>:
>>
>> > Anand Patil wrote:
>> > > On Wed, Nov 5, 2008 at 5:59 PM, Martyn Plummer <[EMAIL PROTECTED]>
>> wrote:
>> > >
>> > >> It looks like the em64t version of MKL fails the test for the
>> accuracy
>> > >> of zdotu ("checking whether double complex BLAS can be used") and is
>> > >> therefore dropped in favour of R's built-in BLAS. I have just tested
>> > >> this on Fedora and get the same result.
>> > >>
>> > >> The 32-bit MKL does work for me.
>> > >>
>> > >> Martyn
>> > >>
>> > >
>> > > Many thanks, Martyn. The 64-bit index space is important to me, will I
>> lose
>> > > it if I link against the 32-bit MKL? Also, should I file a bug report
>> with
>> > > Intel?
>> > > Anand
>> >
>> > There's also the option of "breaking the thermometer". You might examine
>> > that check and decide whether the loss of accuracy is enough for you to
>> > worry about and if not, take out the test from configure.
>> >
>> > Apparently, this check was put in place in R-2.2.0
>> >
>> > o Any external BLAS found is now tested to see if the complex
>> > routine zdotu works correctly: this provides a compatibility
>> > test of compiler return conventions.
>> >
>> > which suggests that the expected failure is catastrophic, and looking at
>> > the code, there's a fuzz of 1e-10 which would seem to be about 1e5 times
>> > larger than required.
>>
>> My apologies. I skimmed the code of the test program, but as Brian says
>> it is not a question of accuracy. The test program does in fact segfault,
>> although you won't see this in your config.log.
>>
>> Anyway, Anand, I would just carry on with libRblas.
>>
>> Martyn
>>
>>
>>
>> ---
>> This message and its attachments are strictly confidential. If you are
>> not the intended recipient of this message, please immediately notify
>> the sender and delete it. Since its integrity cannot be guaranteed,
>> its content cannot involve the sender's responsibility. Any misuse,
>> any disclosure or publication of its content, either whole or partial,
>> is prohibited, exception made of formally approved use
>> ---
>>
>
>
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Building with MKL on Ubuntu
On Thu, Nov 6, 2008 at 6:48 AM, Prof Brian Ripley <[EMAIL PROTECTED]>wrote: > And how does that help you? At some future point on a real problem you > will get incorrect answers/segfault. > > That test was put there as a result of such incorrect results found the > hard way. > I appreciate that... but I never use the complex blas, so in my case the test is preventing me from taking advantage of MKL's real blas. Or is the zdotu error symptomatic of a problem that would affect the real blas also? Anand [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Building with MKL on Ubuntu
On Thu, Nov 6, 2008 at 2:01 AM, Ei-ji Nakama <[EMAIL PROTECTED]> wrote:
> Hi
>
> If you want to use MKL10(no 8 or 9), you should obey instructions of
> R-admin.
> MKL10 needs openmp. This needs gcc4.2 or later.
> OpenMP for Intel was necessary with MKL10.0.1.x (There was not it in a
> manual of MKL).
> # Please try an additional effect of "-liomp5" in the %*%
>
> # fp-model of Linux of IA32 is 387(80bit) about the precision,
> # but SSE2(64bit) is a default in Intel64.
> # It is caused by this that results are different.
>
> There was not the problem in R-2.8.0 either
> MKL_LIB_PATH=/opt/intel/mkl/10.0.5.025/lib/em64t
> MKL=" -L${MKL_LIB_PATH} \
> -Wl,--start-group \
> ${MKL_LIB_PATH}/libmkl_gf_lp64.a\
> ${MKL_LIB_PATH}/libmkl_gnu_thread.a \
> ${MKL_LIB_PATH}/libmkl_core.a \
> -Wl,--end-group \
> -liomp5 -lguide -lpthread -lgomp"
> ./configure --with-lapack="$MKL" --with-blas="$MKL"
Thanks Ei-Ji,
I tried that earlier, and it doesn't encounter the zdotu error (maybe it was
fixed in 10.0.5?) but I got a different error:
gcc -std=gnu99 -I../../src/extra/zlib -I../../src/extra/bzip2
-I../../src/extra/pcre -I. -I../../src/include -I../../src/include
-I/usr/local/include -DHAVE_CONFIG_H -fpic -g -O2 -c Rmain.c -o Rmain.o
gcc -std=gnu99 -Wl,--export-dynamic -L/usr/local/lib64 -o R.bin Rmain.o
-L../../lib -lR
/opt/intel/fce/10.1.018/lib/libiomp5.so: undefined reference to
`pthread_atfork'
collect2: ld returned 1 exit status
make[3]: *** [R.bin] Error 1
make[3]: Leaving directory `/working_copies/R/src/main'
make[2]: *** [R] Error 2
However, I saw the same error with 10.0.2 and the broken thermometer;
setting the environment variables listed previously:
CFLAGS=-pthread -O3
FC=gfortran -pthread
FFLAGS=-pthread
CXXFLAGS=-O3 -pthread
FCLAGS=-pthread
LDFLAGS=-lpthread
makes that error not happen, so it looks like we have a real solution!
Anand
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [R] Building with MKL on Ubuntu
On Thu, Nov 6, 2008 at 3:08 PM, Ei-ji Nakama <[EMAIL PROTECTED]> wrote:
> Hi
>
> 2008/11/6 Anand Patil <[EMAIL PROTECTED]>:
> > I tried that earlier, and it doesn't encounter the zdotu error (maybe it
> was
> > fixed in 10.0.5?) but I got a different error:
>
> no, see MKL manual.
> Because fortran do not have compatibility, You appoint ABI of your
> Fortran in a link of MKL.
>
> > /opt/intel/fce/10.1.018/lib/libiomp5.so: undefined reference to
> > `pthread_atfork'
>
> MKL 10.0.1.x that libiomp5 was necessary.
> Hmm, but not need libiomp5 with MKL 10.0.5.x.
> I do not understand which version MKL were recovered in, but libiomp5
> seems to be unnecessary in the latest edition.
>
> MKL_LIB_PATH=/opt/intel/mkl/10.0.5.025/lib/em64t
> MKL=" -L${MKL_LIB_PATH} \
>-Wl,--start-group \
>${MKL_LIB_PATH}/libmkl_gf_lp64.a\
>${MKL_LIB_PATH}/libmkl_gnu_thread.a \
>${MKL_LIB_PATH}/libmkl_core.a \
> -Wl,--end-group -lgomp"
> LDFLAGS="-L$MKL_LIB_PATH" ./configure --with-lapack="$MKL"
> --with-blas="$MKL"
>
> -liomp5 -lguide -lpthread You omit three library, and please link
>
> Please check the result of the big %*% just to make sure.
> --
> EI-JI Nakama
> "\u4e2d\u9593\u6804\u6cbb"
>
That worked for me too. The big %*% does indeed use 800% CPU for a while.
Thanks,
Anand
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] NY Times article
Sorry if this is spam, but I couldn't see it having popped up on the list yet. http://www.nytimes.com/2009/01/07/technology/business-computing/07program.html?emc=eta1 Anand [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
