Add --show to the failing nvcc_wrapper command and see how nvcc_wrapper parses the options
PATH=/global/common/sw/cray/sles15/x86_64/zlib/1.2.11/gcc/8.2.0/pep2pal/bin:/global/common/sw/cray/cnl7/haswell/cmake/3.18.2/bin:/opt/cray/rca/2.2.20-7.0.1.1_4.62__g8e3fb5b.ari/bin:/opt/cray/alps/6.6.58-7.0.1.1_6.20__g437d88db.ari/sbin:/opt/cray/alps/default/bin:/opt/cray/job/2.2.4-7.0.1.1_3.48__g36b56f4.ari/bin:/opt/cray/pe/craype/2.6.2/bin:/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/compilers/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/cuda/bin:/usr/common/software/sles15_cgpu/cuda/11.1.1/bin:/usr/common/software/sles15_cgpu/gcc/8.3.0/bin:/opt/esslurm/bin:/opt/cray/pe/modules/ 3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:`dirname <http://3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:dirname> /usr/common/software/sles15_cgpu/cuda/11.1.1/bin/nvcc` NVCC_WRAPPER_DEFAULT_COMPILER=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin/mpicxx /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/bin/nvcc_wrapper --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode arch=compute_80,code=sm_80 -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -Wno-deprecated-gpu-targets -I/global/homes/m/madams/petsc/include -I/global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/include -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -I/usr/common/software/sles15_cgpu/cuda/11.1.1/include /global/u2/m/madams/petsc/src/sys/objects/kokkos/kinit.kokkos.cxx -o arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o --show --Junchao Zhang On Sat, May 15, 2021 at 7:14 PM Mark Adams <mfad...@lbl.gov> wrote: > Thanks, > Now I get an error in make all. THis was working a few weeks ago. > Make.log was empty but here is the output. > .... > CC arch-cori-gpu-opt-kokkos-gcc/obj/sys/error/ftn-custom/zerrf.o > CC arch-cori-gpu-opt-kokkos-gcc/obj/sys/f90-src/f90_cwrap.o > CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/hdf5io.o > > PATH=/global/common/sw/cray/sles15/x86_64/zlib/1.2.11/gcc/8.2.0/pep2pal/bin:/global/common/sw/cray/cnl7/haswell/cmake/3.18.2/bin:/opt/cray/rca/2.2.20-7.0.1.1_4.62__g8e3fb5b.ari/bin:/opt/cray/alps/6.6.58-7.0.1.1_6.20__g437d88db.ari/sbin:/opt/cray/alps/default/bin:/opt/cray/job/2.2.4-7.0.1.1_3.48__g36b56f4.ari/bin:/opt/cray/pe/craype/2.6.2/bin:/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/compilers/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/cuda/bin:/usr/common/software/sles15_cgpu/cuda/11.1.1/bin:/usr/common/software/sles15_cgpu/gcc/8.3.0/bin:/opt/esslurm/bin:/opt/cray/pe/modules/ > 3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:`dirname > <http://3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:dirname> > /usr/common/software/sles15_cgpu/cuda/11.1.1/bin/nvcc` > NVCC_WRAPPER_DEFAULT_COMPILER=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin/mpicxx > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/bin/nvcc_wrapper > --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2 > -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 > -Xcompiler -fPIC -O3 -gencode arch=compute_80,code=sm_80 > -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include > -Wno-deprecated-gpu-targets -I/global/homes/m/madams/petsc/include > -I/global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/include > -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include > -I/usr/common/software/sles15_cgpu/cuda/11.1.1/include > /global/u2/m/madams/petsc/src/sys/objects/kokkos/kinit.kokkos.cxx -o > arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o > g++: error: arch=compute_80: No such file or directory > g++: error: code=sm_80: No such file or directory > gmake[3]: *** [gmakefile:188: > arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o] Error 1 > gmake[3]: *** Waiting for unfinished jobs.... > CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/isio.o > CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/ftn-auto/pmapf.o > CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/pmap.o > CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/psort.o > CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/isltog.o > FC arch-cori-gpu-opt-kokkos-gcc/obj/vec/f90-mod/petscvecmod.o > CUDAC arch-cori-gpu-opt-kokkos-gcc/obj/sys/memory/cuda/mcudahost.o > CUDAC.dep arch-cori-gpu-opt-kokkos-gcc/obj/sys/memory/cuda/mcudahost.o > gmake[2]: *** [/global/homes/m/madams/petsc/lib/petsc/conf/rules:50: libs] > Error 2 > **************************ERROR************************************* > Error during compile, check > arch-cori-gpu-opt-kokkos-gcc/lib/petsc/conf/make.log > Send it and arch-cori-gpu-opt-kokkos-gcc/lib/petsc/conf/configure.log to > petsc-ma...@mcs.anl.gov > ******************************************************************** > gmake[1]: *** [makefile:40: all] Error 1 > make: *** [GNUmakefile:9: all] Error 2 > > On Sat, May 15, 2021 at 5:30 PM Barry Smith <bsm...@petsc.dev> wrote: > >> >> Bitbucket is off-line. Do you need fortran stubs at this moment? If not >> use --with-fortran-interface=0 >> >> >> On May 15, 2021, at 2:51 PM, Mark Adams <mfad...@lbl.gov> wrote: >> >> I can build a non-kokkos PETSc here (Cori/GPU) but I get this error with >> Kokkos. >> Any suggestions? >> Thanks, >> Mark >> >> ============================================================================================= >> >> >> >> ============================================================================================= >> >> >> Trying to >> download git://https://bitbucket.org/petsc/pkg-sowing.git for SOWING >> >> >> >> >> ============================================================================================= >> >> >> >> ============================================================================================= >> >> >> Trying to >> download git://https://bitbucket.org/petsc/pkg-sowing.git for SOWING >> >> >> >> >> ============================================================================================= >> >> >> >> ============================================================================================= >> >> >> Trying to >> download https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz >> for SOWING >> >> >> >> ============================================================================================= >> >> >> >> >> >> >> >> ******************************************************************************* >> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for >> details): >> >> ------------------------------------------------------------------------------- >> Error during download/extract/detection of SOWING: >> Unable to clone sowing >> Could not execute "['git clone https://bitbucket.org/petsc/pkg-sowing.git >> /global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing']": >> Cloning into >> '/global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing'... >> fatal: unable to access 'https://bitbucket.org/petsc/pkg-sowing.git/': >> The requested URL returned error: 503 >> Unable to download package SOWING from: git:// >> https://bitbucket.org/petsc/pkg-sowing.git >> * If URL specified manually - perhaps there is a typo? >> * If your network is disconnected - please reconnect and rerun ./configure >> * Or perhaps you have a firewall blocking the download >> * You can run with --with-packages-download-dir=/adirectory and >> ./configure will instruct you what packages to download manually >> * or you can download the above URL manually, to /yourselectedlocation >> and use the configure option: >> --download-sowing=/yourselectedlocation >> Unable to clone sowing >> Could not execute "['git clone https://bitbucket.org/petsc/pkg-sowing.git >> /global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing']": >> Cloning into >> '/global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing'... >> fatal: unable to access 'https://bitbucket.org/petsc/pkg-sowing.git/': >> The requested URL returned error: 503 >> Unable to download package SOWING from: git:// >> https://bitbucket.org/petsc/pkg-sowing.git >> * If URL specified manually - perhaps there is a typo? >> * If your network is disconnected - please reconnect and rerun ./configure >> * Or perhaps you have a firewall blocking the download >> * You can run with --with-packages-download-dir=/adirectory and >> ./configure will instruct you what packages to download manually >> * or you can download the above URL manually, to /yourselectedlocation >> and use the configure option: >> --download-sowing=/yourselectedlocation >> file could not be opened successfully >> Downloaded package SOWING from: >> https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz is not a >> tarball. >> [or installed python cannot process compressed files] >> * If you are behind a firewall - please fix your proxy and rerun >> ./configure >> For example at LANL you may need to set the environmental variable >> http_proxy (or HTTP_PROXY?) to http://proxyout.lanl.gov >> * You can run with --with-packages-download-dir=/adirectory and >> ./configure will instruct you what packages to download manually >> * or you can download the above URL manually, to >> /yourselectedlocation/v1.1.26-p1.tar.gz >> and use the configure option: >> --download-sowing=/yourselectedlocation/v1.1.26-p1.tar.gz >> >> ******************************************************************************* >> <configure.log> >> >> >>
