Mark,
The kokkos-kernels did not get rebuilt so they will likely have a different
geodearch.
I think you need to rm -rf arch-cori-gpu-opt-kokkos-gcc
but was --with-mpi-dir=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc
built with CUDA aware MPI and a different gencodarch than is supported by the
hardware? If so, I guess you need to build PETSc using
-with-cuda-gencodearch=70 if that is what the MPI was built with.
Barry
> On May 16, 2021, at 6:13 AM, Mark Adams <mfad...@lbl.gov> wrote:
>
> progress:
>
> 04:08 cgpu11 adams/cusparse-cpu-solve *= ~/petsc$ make
> PETSC_DIR=/global/homes/m/madams/petsc
> PETSC_ARCH=arch-cori-gpu-opt-kokkos-gcc check
> Running check examples to verify correct installation
> Using PETSC_DIR=/global/homes/m/madams/petsc and
> PETSC_ARCH=arch-cori-gpu-opt-kokkos-gcc
> C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process
> Possible error running C/C++ src/snes/tutorials/ex19 with 2 MPI processes
> See http://www.mcs.anl.gov/petsc/documentation/faq.html
> <http://www.mcs.anl.gov/petsc/documentation/faq.html>
> srun: error: Unable to create step for job 1921749: More processors requested
> than permitted
> 2,15c2,32
> < 0 SNES Function norm 2.391552133017e-01
> < 0 KSP Residual norm 2.325621076120e-01
> < 1 KSP Residual norm 1.654206318674e-02
> < 2 KSP Residual norm 7.202836119880e-04
> < 3 KSP Residual norm 1.796861424199e-05
> < 4 KSP Residual norm 2.461332992052e-07
> < 1 SNES Function norm 6.826585648929e-05
> < 0 KSP Residual norm 2.347339172985e-05
> < 1 KSP Residual norm 8.356798075993e-07
> < 2 KSP Residual norm 1.844045309619e-08
> < 3 KSP Residual norm 5.336386977405e-10
> < 4 KSP Residual norm 2.662608472862e-11
> < 2 SNES Function norm 6.549682264799e-11
> < Number of SNES iterations = 2
> ---
> > Kokkos::Cuda::initialize ERROR: likely mismatch of architecture
> > [cgpu11:39927] *** Process received signal ***
> > [cgpu11:39927] Signal: Aborted (6)
> > [cgpu11:39927] Signal code: (-6)
> > [cgpu11:39927] [ 0] /lib64/libpthread.so.0(+0x12360)[0x2aab12731360]
> > [cgpu11:39927] [ 1] /lib64/libc.so.6(gsignal+0x110)[0x2aab12973160]
> > [cgpu11:39927] [ 2] /lib64/libc.so.6(abort+0x151)[0x2aab12974741]
> > [cgpu11:39927] [ 3]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libkokkoscore.so.3.4(+0x408e5)[0x2aaabbfb88e5]
> > [cgpu11:39927] [ 4]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libkokkoscore.so.3.4(_ZN6Kokkos4Impl12CudaInternal10initializeEiP11CUstream_st+0xb5c)[0x2aaabbfc762c]
> > [cgpu11:39927] [ 5]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libkokkoscore.so.3.4(_ZN6Kokkos4Impl16ExecSpaceManager17initialize_spacesERKNS_13InitArgumentsE+0x24)[0x2aaabbfa1224]
> > [cgpu11:39927] [ 6]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libkokkoscore.so.3.4(_ZN6Kokkos4Impl70_GLOBAL__N__46_tmpxft_00011209_00000000_7_Kokkos_Core_cpp1_ii_889c95a619initialize_internalERKNS_13InitArgumentsE+0x11)[0x2aaabbfa49d1]
> > [cgpu11:39927] [ 7]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(PetscKokkosInitializeCheck+0x1f9)[0x2aaaaafad809]
> > [cgpu11:39927] [ 8]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x68541b)[0x2aaaab35841b]
> > [cgpu11:39927] [ 9]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x36a224)[0x2aaaab03d224]
> > [cgpu11:39927] [10]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x35f596)[0x2aaaab032596]
> > [cgpu11:39927] [11]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x2f9d94)[0x2aaaaafccd94]
> > [cgpu11:39927] [12]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(PetscSFBcastWithMemTypeBegin+0x11f)[0x2aaaab04bc3f]
> > [cgpu11:39927] [13]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x37f53a)[0x2aaaab05253a]
> > [cgpu11:39927] [14]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(VecScatterBegin+0x70)[0x2aaaab057b30]
> > [cgpu11:39927] [15]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(DMGlobalToLocalBegin_DA+0x1f)[0x2aaaabc27a5f]
> > [cgpu11:39927] [16]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(DMGlobalToLocalBegin+0x1dd)[0x2aaaabe61b3d]
> > [cgpu11:39927] [17]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x144567a)[0x2aaaac11867a]
> > [cgpu11:39927] [18]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(SNESComputeFunction+0xf5)[0x2aaaac138ef5]
> > [cgpu11:39927] [19]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(+0x14ad0de)[0x2aaaac1800de]
> > [cgpu11:39927] [20]
> > /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/lib/libpetsc.so.3.015(SNESSolve+0x821)[0x2aaaac146ed1]
> > [cgpu11:39927] [21]
> > /global/u2/m/madams/petsc/src/snes/tutorials/./ex19[0x401a69]
> > [cgpu11:39927] [22] /lib64/libc.so.6(__libc_start_main+0xea)[0x2aab1295df8a]
> > [cgpu11:39927] [23]
> > /global/u2/m/madams/petsc/src/snes/tutorials/./ex19[0x4021ba]
> > [cgpu11:39927] *** End of error message ***
> > srun: error: cgpu11: task 0: Aborted
> > srun: Terminating job step 1921749.6
> /global/homes/m/madams/petsc/src/snes/tutorials
> Possible problem with ex19 running with cuda, diffs above
> =========================================
> gmake[3]: [makefile:102: runex3k_kokkos] Error 1 (ignored)
>
> On Sun, May 16, 2021 at 12:31 AM Barry Smith <bsm...@petsc.dev
> <mailto:bsm...@petsc.dev>> wrote:
>
> Looks like nvcc_wrapper gets confused when both -arch and -gencode are
> used. But handles either of them separately correctly.
>
> Mark,
>
> PETSc configure now handles the -arch and --with-kokkos-cuda-arch=
> business automatically so you do not, and generally shouldn't pass -arch
> sm_70 in --CUDAFLAGS anymore or use the --with-kokkos-cuda-arch option
> anymore.
>
> If you need to set the -arch to something earlier than the system
> supports (rare) you can now use -with-cuda-gencodearch to set that instead of
> using CUDAFLAGS.
>
> Barry
>
>
> $ NVCC_WRAPPER_DEFAULT_COMPILER=mpicxx
> /home/bsmith/petsc/arch-main/bin/nvcc_wrapper --show --expt-extended-lambda
> -c -Xcompiler -fPIC -g -gencode arch=compute_70,code=sm_70
> -I/home/bsmith/soft/gnu-mpich/include -Wno-deprecated-gpu-targets
> -I/home/bsmith/petsc/include -I/home/bsmith/petsc/arch-main/include
> -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include
> /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx
> -o arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
> nvcc --expt-extended-lambda -gencode arch=compute_70,code=sm_70
> -Wno-deprecated-gpu-targets -ccbin mpicxx -g
> -I/home/bsmith/soft/gnu-mpich/include -I/home/bsmith/petsc/include
> -I/home/bsmith/petsc/arch-main/include
> -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include
> -Xcompiler -fPIC -x cu
> /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx -c -o
> arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
> ~/petsc (main=) arch-main
>
> $ NVCC_WRAPPER_DEFAULT_COMPILER=mpicxx
> /home/bsmith/petsc/arch-main/bin/nvcc_wrapper --show --expt-extended-lambda
> -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10
> -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler -fPIC -O3 -gencode
> arch=compute_80,code=sm_80
> -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include
> -Wno-deprecated-gpu-targets
> nvcc --expt-extended-lambda -arch=sm_70 -Wno-deprecated-gpu-targets -ccbin
> mpicxx -DLANDAU_DIM=2 -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC
> -DLANDAU_MAX_Q=4 -O3
> -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include -Xcompiler
> -rdynamic,-fPIC,arch=compute_80,code=sm_80 -c
> ~/petsc (main=) arch-main
>
> $ NVCC_WRAPPER_DEFAULT_COMPILER=mpicxx
> /home/bsmith/petsc/arch-main/bin/nvcc_wrapper --show --expt-extended-lambda
> -c -Xcompiler -fPIC -g -arch=sm_70 -gencode arch=compute_70,code=sm_70
> -I/home/bsmith/soft/gnu-mpich/include -Wno-deprecated-gpu-targets
> -I/home/bsmith/petsc/include -I/home/bsmith/petsc/arch-main/include
> -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include
> /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx
> -o arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
> nvcc --expt-extended-lambda -arch=sm_70 -Wno-deprecated-gpu-targets -ccbin
> mpicxx -g -I/home/bsmith/soft/gnu-mpich/include -I/home/bsmith/petsc/include
> -I/home/bsmith/petsc/arch-main/include
> -I/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/include
> -Xcompiler -fPIC,arch=compute_70,code=sm_70 -x cu
> /home/bsmith/petsc/src/ts/utils/dmplexlandau/kokkos/landau.kokkos.cxx -c -o
> arch-main/obj/ts/utils/dmplexlandau/kokkos/landau.o
>
>
>> On May 15, 2021, at 8:44 PM, Junchao Zhang <junchao.zh...@gmail.com
>> <mailto:junchao.zh...@gmail.com>> wrote:
>>
>> Add --show to the failing nvcc_wrapper command and see how nvcc_wrapper
>> parses the options
>>
>> PATH=/global/common/sw/cray/sles15/x86_64/zlib/1.2.11/gcc/8.2.0/pep2pal/bin:/global/common/sw/cray/cnl7/haswell/cmake/3.18.2/bin:/opt/cray/rca/2.2.20-7.0.1.1_4.62__g8e3fb5b.ari/bin:/opt/cray/alps/6.6.58-7.0.1.1_6.20__g437d88db.ari/sbin:/opt/cray/alps/default/bin:/opt/cray/job/2.2.4-7.0.1.1_3.48__g36b56f4.ari/bin:/opt/cray/pe/craype/2.6.2/bin:/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/compilers/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/cuda/bin:/usr/common/software/sles15_cgpu/cuda/11.1.1/bin:/usr/common/software/sles15_cgpu/gcc/8.3.0/bin:/opt/esslurm/bin:/opt/cray/pe/modules/3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:`dirname
>>
>> <http://3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:dirname>
>> /usr/common/software/sles15_cgpu/cuda/11.1.1/bin/nvcc`
>> NVCC_WRAPPER_DEFAULT_COMPILER=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin/mpicxx
>> /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/bin/nvcc_wrapper
>> --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2
>> -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler
>> -fPIC -O3 -gencode arch=compute_80,code=sm_80
>> -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include
>> -Wno-deprecated-gpu-targets -I/global/homes/m/madams/petsc/include
>> -I/global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/include
>> -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include
>> -I/usr/common/software/sles15_cgpu/cuda/11.1.1/include
>> /global/u2/m/madams/petsc/src/sys/objects/kokkos/kinit.kokkos.cxx -o
>> arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o --show
>>
>> --Junchao Zhang
>>
>>
>> On Sat, May 15, 2021 at 7:14 PM Mark Adams <mfad...@lbl.gov
>> <mailto:mfad...@lbl.gov>> wrote:
>> Thanks,
>> Now I get an error in make all. THis was working a few weeks ago.
>> Make.log was empty but here is the output.
>> ....
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/sys/error/ftn-custom/zerrf.o
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/sys/f90-src/f90_cwrap.o
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/hdf5io.o
>> PATH=/global/common/sw/cray/sles15/x86_64/zlib/1.2.11/gcc/8.2.0/pep2pal/bin:/global/common/sw/cray/cnl7/haswell/cmake/3.18.2/bin:/opt/cray/rca/2.2.20-7.0.1.1_4.62__g8e3fb5b.ari/bin:/opt/cray/alps/6.6.58-7.0.1.1_6.20__g437d88db.ari/sbin:/opt/cray/alps/default/bin:/opt/cray/job/2.2.4-7.0.1.1_3.48__g36b56f4.ari/bin:/opt/cray/pe/craype/2.6.2/bin:/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/compilers/bin:/usr/common/software/sles15_cgpu/hpcsdk/20.11/Linux_x86_64/20.11/cuda/bin:/usr/common/software/sles15_cgpu/cuda/11.1.1/bin:/usr/common/software/sles15_cgpu/gcc/8.3.0/bin:/opt/esslurm/bin:/opt/cray/pe/modules/3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:`dirname
>>
>> <http://3.2.11.4/bin:/usr/common/software/bin:/usr/common/nsg/bin:/opt/ovis/bin:/opt/ovis/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/cray/pe/bin:dirname>
>> /usr/common/software/sles15_cgpu/cuda/11.1.1/bin/nvcc`
>> NVCC_WRAPPER_DEFAULT_COMPILER=/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/bin/mpicxx
>> /global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/bin/nvcc_wrapper
>> --expt-extended-lambda -c -arch=sm_70 -Xcompiler -rdynamic -DLANDAU_DIM=2
>> -DLANDAU_MAX_SPECIES=10 -DPETSC_HAVE_CUDA_ATOMIC -DLANDAU_MAX_Q=4 -Xcompiler
>> -fPIC -O3 -gencode arch=compute_80,code=sm_80
>> -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include
>> -Wno-deprecated-gpu-targets -I/global/homes/m/madams/petsc/include
>> -I/global/homes/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/include
>> -I/usr/common/software/sles15_cgpu/openmpi/4.0.3/gcc/include
>> -I/usr/common/software/sles15_cgpu/cuda/11.1.1/include
>> /global/u2/m/madams/petsc/src/sys/objects/kokkos/kinit.kokkos.cxx -o
>> arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o
>> g++: error: arch=compute_80: No such file or directory
>> g++: error: code=sm_80: No such file or directory
>> gmake[3]: *** [gmakefile:188:
>> arch-cori-gpu-opt-kokkos-gcc/obj/sys/objects/kokkos/kinit.o] Error 1
>> gmake[3]: *** Waiting for unfinished jobs....
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/isio.o
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/ftn-auto/pmapf.o
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/pmap.o
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/psort.o
>> CC arch-cori-gpu-opt-kokkos-gcc/obj/vec/is/utils/isltog.o
>> FC arch-cori-gpu-opt-kokkos-gcc/obj/vec/f90-mod/petscvecmod.o
>> CUDAC arch-cori-gpu-opt-kokkos-gcc/obj/sys/memory/cuda/mcudahost.o
>> CUDAC.dep arch-cori-gpu-opt-kokkos-gcc/obj/sys/memory/cuda/mcudahost.o
>> gmake[2]: *** [/global/homes/m/madams/petsc/lib/petsc/conf/rules:50: libs]
>> Error 2
>> **************************ERROR*************************************
>> Error during compile, check
>> arch-cori-gpu-opt-kokkos-gcc/lib/petsc/conf/make.log
>> Send it and arch-cori-gpu-opt-kokkos-gcc/lib/petsc/conf/configure.log to
>> petsc-ma...@mcs.anl.gov <mailto:petsc-ma...@mcs.anl.gov>
>> ********************************************************************
>> gmake[1]: *** [makefile:40: all] Error 1
>> make: *** [GNUmakefile:9: all] Error 2
>>
>> On Sat, May 15, 2021 at 5:30 PM Barry Smith <bsm...@petsc.dev
>> <mailto:bsm...@petsc.dev>> wrote:
>>
>> Bitbucket is off-line. Do you need fortran stubs at this moment? If not
>> use --with-fortran-interface=0
>>
>>
>>> On May 15, 2021, at 2:51 PM, Mark Adams <mfad...@lbl.gov
>>> <mailto:mfad...@lbl.gov>> wrote:
>>>
>>> I can build a non-kokkos PETSc here (Cori/GPU) but I get this error with
>>> Kokkos.
>>> Any suggestions?
>>> Thanks,
>>> Mark
>>>
>>> =============================================================================================
>>>
>>>
>>>
>>> =============================================================================================
>>>
>>>
>>> Trying to
>>> download git://https://bitbucket.org/petsc/pkg-sowing.git
>>> <https://bitbucket.org/petsc/pkg-sowing.git> for SOWING
>>>
>>>
>>>
>>> =============================================================================================
>>>
>>>
>>>
>>> =============================================================================================
>>>
>>>
>>> Trying to
>>> download git://https://bitbucket.org/petsc/pkg-sowing.git
>>> <https://bitbucket.org/petsc/pkg-sowing.git> for SOWING
>>>
>>>
>>>
>>> =============================================================================================
>>>
>>>
>>>
>>> =============================================================================================
>>>
>>>
>>> Trying to
>>> download https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz
>>> <https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz> for SOWING
>>>
>>>
>>>
>>> =============================================================================================
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *******************************************************************************
>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for
>>> details):
>>> -------------------------------------------------------------------------------
>>> Error during download/extract/detection of SOWING:
>>> Unable to clone sowing
>>> Could not execute "['git clone https://bitbucket.org/petsc/pkg-sowing.git
>>> <https://bitbucket.org/petsc/pkg-sowing.git>
>>> /global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing']":
>>> Cloning into
>>> '/global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing'...
>>> fatal: unable to access 'https://bitbucket.org/petsc/pkg-sowing.git/
>>> <https://bitbucket.org/petsc/pkg-sowing.git/>': The requested URL returned
>>> error: 503
>>> Unable to download package SOWING from:
>>> git://https://bitbucket.org/petsc/pkg-sowing.git
>>> <https://bitbucket.org/petsc/pkg-sowing.git>
>>> * If URL specified manually - perhaps there is a typo?
>>> * If your network is disconnected - please reconnect and rerun ./configure
>>> * Or perhaps you have a firewall blocking the download
>>> * You can run with --with-packages-download-dir=/adirectory and ./configure
>>> will instruct you what packages to download manually
>>> * or you can download the above URL manually, to /yourselectedlocation
>>> and use the configure option:
>>> --download-sowing=/yourselectedlocation
>>> Unable to clone sowing
>>> Could not execute "['git clone https://bitbucket.org/petsc/pkg-sowing.git
>>> <https://bitbucket.org/petsc/pkg-sowing.git>
>>> /global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing']":
>>> Cloning into
>>> '/global/u2/m/madams/petsc/arch-cori-gpu-opt-kokkos-gcc/externalpackages/git.sowing'...
>>> fatal: unable to access 'https://bitbucket.org/petsc/pkg-sowing.git/
>>> <https://bitbucket.org/petsc/pkg-sowing.git/>': The requested URL returned
>>> error: 503
>>> Unable to download package SOWING from:
>>> git://https://bitbucket.org/petsc/pkg-sowing.git
>>> <https://bitbucket.org/petsc/pkg-sowing.git>
>>> * If URL specified manually - perhaps there is a typo?
>>> * If your network is disconnected - please reconnect and rerun ./configure
>>> * Or perhaps you have a firewall blocking the download
>>> * You can run with --with-packages-download-dir=/adirectory and ./configure
>>> will instruct you what packages to download manually
>>> * or you can download the above URL manually, to /yourselectedlocation
>>> and use the configure option:
>>> --download-sowing=/yourselectedlocation
>>> file could not be opened successfully
>>> Downloaded package SOWING from:
>>> https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz
>>> <https://bitbucket.org/petsc/pkg-sowing/get/v1.1.26-p1.tar.gz> is not a
>>> tarball.
>>> [or installed python cannot process compressed files]
>>> * If you are behind a firewall - please fix your proxy and rerun ./configure
>>> For example at LANL you may need to set the environmental variable
>>> http_proxy (or HTTP_PROXY?) to http://proxyout.lanl.gov
>>> <http://proxyout.lanl.gov/>
>>> * You can run with --with-packages-download-dir=/adirectory and ./configure
>>> will instruct you what packages to download manually
>>> * or you can download the above URL manually, to
>>> /yourselectedlocation/v1.1.26-p1.tar.gz
>>> and use the configure option:
>>> --download-sowing=/yourselectedlocation/v1.1.26-p1.tar.gz
>>> *******************************************************************************
>>> <configure.log>
>>
>
> <make.log><configure.log>
