Source: gromacs Version: 2020.4-2 Severity: serious Justification: FTBFS on amd64 Tags: bullseye sid ftbfs Usertags: ftbfs-20210120 ftbfs-bullseye
Hi, During a rebuild of all packages in sid, your package failed to build on amd64. Relevant part (hopefully): > make[4]: Entering directory '/<<PKGBUILDDIR>>/build/mpich' > make[4]: Nothing to be done for 'CMakeFiles/tests.dir/build'. > make[4]: Leaving directory '/<<PKGBUILDDIR>>/build/mpich' > [100%] Built target tests > make[3]: Leaving directory '/<<PKGBUILDDIR>>/build/mpich' > /usr/bin/cmake -E cmake_progress_start > /<<PKGBUILDDIR>>/build/mpich/CMakeFiles 0 > make[2]: Leaving directory '/<<PKGBUILDDIR>>/build/mpich' > make[1]: Leaving directory '/<<PKGBUILDDIR>>/build/mpich' > (cd build/mpich; LD_LIBRARY_PATH=/<<PKGBUILDDIR>>/build/mpich/lib \ > ctest -V || dpkg-architecture -i hurd-i386 || > dpkg-architecture -i armhf ) > UpdateCTestConfiguration from > :/<<PKGBUILDDIR>>/build/mpich/DartConfiguration.tcl > Parse Config file:/<<PKGBUILDDIR>>/build/mpich/DartConfiguration.tcl > UpdateCTestConfiguration from > :/<<PKGBUILDDIR>>/build/mpich/DartConfiguration.tcl > Parse Config file:/<<PKGBUILDDIR>>/build/mpich/DartConfiguration.tcl > Test project /<<PKGBUILDDIR>>/build/mpich > Constructing a list of tests > Done constructing a list of tests > Updating test list for fixtures > Added 0 tests to meet fixture requirements > Checking test dependency graph... > Checking test dependency graph end > test 1 > Start 1: TestUtilsUnitTests > > 1: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/TestUtilsUnitTests.xml" > 1: Test timeout computed to be: 30 > 1: [1611165257.171056] [ip-172-31-13-129:15115:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 1: [1611165257.171089] [ip-172-31-13-129:15115:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 1: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 1: MPIR_Init_thread(152).......: > 1: MPID_Init(597)..............: > 1: MPIDI_UCX_mpi_init_hook(247): > 1: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 1: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 1: : > 1: system msg for write_line failure : Bad file descriptor > 1: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 1: MPIR_Init_thread(152).......: > 1: MPID_Init(597)..............: > 1: MPIDI_UCX_mpi_init_hook(247): > 1: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 1: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 1: : > 1: system msg for write_line failure : Bad file descriptor > 1: [ip-172-31-13-129:15115:0:15115] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 1: ==== backtrace (tid: 15115) ==== > 1: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f8398998ea4] > 1: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f83989990af] > 1: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f839899926a] > 1: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f8399997140] > 1: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f83991ca431] > 1: 5 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x122f45) > [0x562fbb939f45] > 1: 6 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0xdf969) > [0x562fbb8f6969] > 1: 7 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x93890) > [0x562fbb8aa890] > 1: 8 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x49efd) > [0x562fbb860efd] > 1: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f83989e5d0a] > 1: 10 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x4a88a) > [0x562fbb86188a] > 1: ================================= > 1/30 Test #1: TestUtilsUnitTests ...............***Exception: SegFault > 0.10 sec > [1611165257.171056] [ip-172-31-13-129:15115:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.171089] [ip-172-31-13-129:15115:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15115:0:15115] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15115) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f8398998ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f83989990af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f839899926a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f8399997140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f83991ca431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x122f45) > [0x562fbb939f45] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0xdf969) [0x562fbb8f6969] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x93890) [0x562fbb8aa890] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x49efd) [0x562fbb860efd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f83989e5d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/testutils-test(+0x4a88a) [0x562fbb86188a] > ================================= > > test 2 > Start 2: TestUtilsMpiUnitTests > > 2: Test command: /usr/bin/mpiexec.mpich "-np" "2" "-host" "localhost" > "/<<PKGBUILDDIR>>/build/mpich/bin/testutils-mpi-test" > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/TestUtilsMpiUnitTests.xml" > 2: Test timeout computed to be: 30 > 2: [1611165257.208556] [ip-172-31-13-129:15125:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 2: [1611165257.208581] [ip-172-31-13-129:15125:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 2: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 2: MPIR_Init_thread(152).......: > 2: MPID_Init(597)..............: > 2: MPIDI_UCX_mpi_init_hook(247): > 2: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 2/30 Test #2: TestUtilsMpiUnitTests ............***Failed 0.03 sec > [1611165257.208556] [ip-172-31-13-129:15125:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.208581] [ip-172-31-13-129:15125:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > > test 3 > Start 3: UtilityUnitTests > > 3: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/utility-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/UtilityUnitTests.xml" > 3: Test timeout computed to be: 30 > 3: [1611165257.232618] [ip-172-31-13-129:15142:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 3: [1611165257.232648] [ip-172-31-13-129:15142:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 3: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 3: MPIR_Init_thread(152).......: > 3: MPID_Init(597)..............: > 3: MPIDI_UCX_mpi_init_hook(247): > 3: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 3: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 3: : > 3: system msg for write_line failure : Bad file descriptor > 3: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 3: MPIR_Init_thread(152).......: > 3: MPID_Init(597)..............: > 3: MPIDI_UCX_mpi_init_hook(247): > 3: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 3: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 3: : > 3: system msg for write_line failure : Bad file descriptor > 3: [ip-172-31-13-129:15142:0:15142] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 3: ==== backtrace (tid: 15142) ==== > 3: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f497c87eea4] > 3: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f497c87f0af] > 3: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f497c87f26a] > 3: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f497d87d140] > 3: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f497d0b0431] > 3: 5 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x223405) > [0x562cc5240405] > 3: 6 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x1df5d9) > [0x562cc51fc5d9] > 3: 7 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x199200) > [0x562cc51b6200] > 3: 8 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x7508d) > [0x562cc509208d] > 3: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f497c8cbd0a] > 3: 10 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x75a9a) > [0x562cc5092a9a] > 3: ================================= > 3/30 Test #3: UtilityUnitTests .................***Exception: SegFault > 0.02 sec > [1611165257.232618] [ip-172-31-13-129:15142:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.232648] [ip-172-31-13-129:15142:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15142:0:15142] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15142) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f497c87eea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f497c87f0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f497c87f26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f497d87d140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f497d0b0431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x223405) [0x562cc5240405] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x1df5d9) [0x562cc51fc5d9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x199200) [0x562cc51b6200] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x7508d) [0x562cc509208d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f497c8cbd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/utility-test(+0x75a9a) [0x562cc5092a9a] > ================================= > > test 4 > Start 4: UtilityMpiUnitTests > > 4: Test command: /usr/bin/mpiexec.mpich "-np" "4" "-host" "localhost" > "/<<PKGBUILDDIR>>/build/mpich/bin/utility-mpi-test" > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/UtilityMpiUnitTests.xml" > 4: Test timeout computed to be: 30 > 4: [1611165257.268625] [ip-172-31-13-129:15154:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 4: [1611165257.268653] [ip-172-31-13-129:15154:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 4: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 4: MPIR_Init_thread(152).......: > 4: MPID_Init(597)..............: > 4: MPIDI_UCX_mpi_init_hook(247): > 4: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 4/30 Test #4: UtilityMpiUnitTests ..............***Failed 0.04 sec > [1611165257.268625] [ip-172-31-13-129:15154:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.268653] [ip-172-31-13-129:15154:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > > test 5 > Start 5: MdlibUnitTest > > 5: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/MdlibUnitTest.xml" > 5: Test timeout computed to be: 30 > 5: [1611165257.294476] [ip-172-31-13-129:15183:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 5: [1611165257.294505] [ip-172-31-13-129:15183:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 5: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 5: MPIR_Init_thread(152).......: > 5: MPID_Init(597)..............: > 5: MPIDI_UCX_mpi_init_hook(247): > 5: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 5: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 5: : > 5: system msg for write_line failure : Bad file descriptor > 5: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 5: MPIR_Init_thread(152).......: > 5: MPID_Init(597)..............: > 5: MPIDI_UCX_mpi_init_hook(247): > 5: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 5: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 5: : > 5: system msg for write_line failure : Bad file descriptor > 5: [ip-172-31-13-129:15183:0:15183] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 5: ==== backtrace (tid: 15183) ==== > 5: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f985e6deea4] > 5: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f985e6df0af] > 5: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f985e6df26a] > 5: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f985f6e3140] > 5: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f985ef16431] > 5: 5 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0x1e4355) [0x558002461355] > 5: 6 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0xfbd89) [0x558002378d89] > 5: 7 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0xcf780) [0x55800234c780] > 5: 8 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0x669ed) [0x5580022e39ed] > 5: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f985e731d0a] > 5: 10 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0x67a8a) [0x5580022e4a8a] > 5: ================================= > 5/30 Test #5: MdlibUnitTest ....................***Exception: SegFault > 0.02 sec > [1611165257.294476] [ip-172-31-13-129:15183:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.294505] [ip-172-31-13-129:15183:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15183:0:15183] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15183) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f985e6deea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f985e6df0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f985e6df26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f985f6e3140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f985ef16431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0x1e4355) [0x558002461355] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0xfbd89) [0x558002378d89] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0xcf780) [0x55800234c780] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0x669ed) [0x5580022e39ed] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f985e731d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/mdlib-test(+0x67a8a) [0x5580022e4a8a] > ================================= > > test 6 > Start 6: AppliedForcesUnitTest > > 6: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/AppliedForcesUnitTest.xml" > 6: Test timeout computed to be: 30 > 6: [1611165257.317635] [ip-172-31-13-129:15191:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 6: [1611165257.317657] [ip-172-31-13-129:15191:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 6: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 6: MPIR_Init_thread(152).......: > 6: MPID_Init(597)..............: > 6: MPIDI_UCX_mpi_init_hook(247): > 6: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 6: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 6: : > 6: system msg for write_line failure : Bad file descriptor > 6: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 6: MPIR_Init_thread(152).......: > 6: MPID_Init(597)..............: > 6: MPIDI_UCX_mpi_init_hook(247): > 6: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 6: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 6: : > 6: system msg for write_line failure : Bad file descriptor > 6: [ip-172-31-13-129:15191:0:15191] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 6: ==== backtrace (tid: 15191) ==== > 6: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f1556be5ea4] > 6: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f1556be60af] > 6: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f1556be626a] > 6: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f1557bea140] > 6: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f155741d431] > 6: 5 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x12ff25) > [0x55cc711fef25] > 6: 6 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0xb7679) > [0x55cc71186679] > 6: 7 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x6a880) > [0x55cc71139880] > 6: 8 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x4cdcd) > [0x55cc7111bdcd] > 6: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f1556c38d0a] > 6: 10 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x4d75a) > [0x55cc7111c75a] > 6: ================================= > 6/30 Test #6: AppliedForcesUnitTest ............***Exception: SegFault > 0.02 sec > [1611165257.317635] [ip-172-31-13-129:15191:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.317657] [ip-172-31-13-129:15191:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15191:0:15191] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15191) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f1556be5ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f1556be60af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f1556be626a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f1557bea140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f155741d431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x12ff25) > [0x55cc711fef25] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0xb7679) > [0x55cc71186679] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x6a880) > [0x55cc71139880] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x4cdcd) > [0x55cc7111bdcd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f1556c38d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/applied_forces-test(+0x4d75a) > [0x55cc7111c75a] > ================================= > > test 7 > Start 7: CommandLineUnitTests > > 7: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/CommandLineUnitTests.xml" > 7: Test timeout computed to be: 30 > 7: [1611165257.340262] [ip-172-31-13-129:15199:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 7: [1611165257.340283] [ip-172-31-13-129:15199:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 7: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 7: MPIR_Init_thread(152).......: > 7: MPID_Init(597)..............: > 7: MPIDI_UCX_mpi_init_hook(247): > 7: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 7: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 7: : > 7: system msg for write_line failure : Bad file descriptor > 7: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 7: MPIR_Init_thread(152).......: > 7: MPID_Init(597)..............: > 7: MPIDI_UCX_mpi_init_hook(247): > 7: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 7: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 7: : > 7: system msg for write_line failure : Bad file descriptor > 7: [ip-172-31-13-129:15199:0:15199] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 7: ==== backtrace (tid: 15199) ==== > 7: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2946613ea4] > 7: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f29466140af] > 7: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f294661426a] > 7: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2947612140] > 7: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2946e45431] > 7: 5 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x154715) > [0x56394f8fd715] > 7: 6 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x10c1b9) > [0x56394f8b51b9] > 7: 7 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0xc0700) > [0x56394f869700] > 7: 8 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x50f8d) > [0x56394f7f9f8d] > 7: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f2946660d0a] > 7: 10 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x5194a) > [0x56394f7fa94a] > 7: ================================= > 7/30 Test #7: CommandLineUnitTests .............***Exception: SegFault > 0.02 sec > [1611165257.340262] [ip-172-31-13-129:15199:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.340283] [ip-172-31-13-129:15199:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15199:0:15199] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15199) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2946613ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f29466140af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f294661426a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2947612140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2946e45431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x154715) > [0x56394f8fd715] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x10c1b9) > [0x56394f8b51b9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0xc0700) > [0x56394f869700] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x50f8d) > [0x56394f7f9f8d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f2946660d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/commandline-test(+0x5194a) > [0x56394f7fa94a] > ================================= > > test 8 > Start 8: DomDecTests > > 8: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/DomDecTests.xml" > 8: Test timeout computed to be: 30 > 8: [1611165257.362231] [ip-172-31-13-129:15207:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 8: [1611165257.362255] [ip-172-31-13-129:15207:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 8: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 8: MPIR_Init_thread(152).......: > 8: MPID_Init(597)..............: > 8: MPIDI_UCX_mpi_init_hook(247): > 8: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 8: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 8: : > 8: system msg for write_line failure : Bad file descriptor > 8: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 8: MPIR_Init_thread(152).......: > 8: MPID_Init(597)..............: > 8: MPIDI_UCX_mpi_init_hook(247): > 8: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 8: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 8: : > 8: system msg for write_line failure : Bad file descriptor > 8: [ip-172-31-13-129:15207:0:15207] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 8: ==== backtrace (tid: 15207) ==== > 8: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fab7f249ea4] > 8: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fab7f24a0af] > 8: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fab7f24a26a] > 8: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fab80248140] > 8: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fab7fa7b431] > 8: 5 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0xc9e05) [0x5606bde2ce05] > 8: 6 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x85559) [0x5606bdde8559] > 8: 7 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x47d40) [0x5606bddaad40] > 8: 8 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x3a80d) [0x5606bdd9d80d] > 8: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fab7f296d0a] > 8: 10 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x3b0ca) [0x5606bdd9e0ca] > 8: ================================= > 8/30 Test #8: DomDecTests ......................***Exception: SegFault > 0.02 sec > [1611165257.362231] [ip-172-31-13-129:15207:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.362255] [ip-172-31-13-129:15207:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15207:0:15207] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15207) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fab7f249ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fab7f24a0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fab7f24a26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fab80248140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fab7fa7b431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0xc9e05) [0x5606bde2ce05] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x85559) [0x5606bdde8559] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x47d40) [0x5606bddaad40] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x3a80d) [0x5606bdd9d80d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fab7f296d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/domdec-test(+0x3b0ca) [0x5606bdd9e0ca] > ================================= > > test 9 > Start 9: EwaldUnitTests > > 9: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/EwaldUnitTests.xml" > 9: Test timeout computed to be: 30 > 9: [1611165257.383961] [ip-172-31-13-129:15215:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 9: [1611165257.383983] [ip-172-31-13-129:15215:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 9: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 9: MPIR_Init_thread(152).......: > 9: MPID_Init(597)..............: > 9: MPIDI_UCX_mpi_init_hook(247): > 9: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 9: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 9: : > 9: system msg for write_line failure : Bad file descriptor > 9: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 9: MPIR_Init_thread(152).......: > 9: MPID_Init(597)..............: > 9: MPIDI_UCX_mpi_init_hook(247): > 9: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 9: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 9: : > 9: system msg for write_line failure : Bad file descriptor > 9: [ip-172-31-13-129:15215:0:15215] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 9: ==== backtrace (tid: 15215) ==== > 9: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f71736daea4] > 9: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f71736db0af] > 9: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f71736db26a] > 9: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f7174707140] > 9: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f7173f3a431] > 9: 5 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0x169f55) [0x55ac066dbf55] > 9: 6 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0xd5289) [0x55ac06647289] > 9: 7 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0xa9d40) [0x55ac0661bd40] > 9: 8 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0x553f0) [0x55ac065c73f0] > 9: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f7173755d0a] > 9: 10 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0x5604a) [0x55ac065c804a] > 9: ================================= > 9/30 Test #9: EwaldUnitTests ...................***Exception: SegFault > 0.02 sec > [1611165257.383961] [ip-172-31-13-129:15215:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.383983] [ip-172-31-13-129:15215:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15215:0:15215] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15215) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f71736daea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f71736db0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f71736db26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f7174707140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f7173f3a431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0x169f55) [0x55ac066dbf55] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0xd5289) [0x55ac06647289] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0xa9d40) [0x55ac0661bd40] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0x553f0) [0x55ac065c73f0] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f7173755d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/ewald-test(+0x5604a) [0x55ac065c804a] > ================================= > > test 10 > Start 10: FFTUnitTests > > 10: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/fft-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/FFTUnitTests.xml" > 10: Test timeout computed to be: 30 > 10: [1611165257.405339] [ip-172-31-13-129:15223:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 10: [1611165257.405363] [ip-172-31-13-129:15223:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 10: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 10: MPIR_Init_thread(152).......: > 10: MPID_Init(597)..............: > 10: MPIDI_UCX_mpi_init_hook(247): > 10: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 10: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 10: : > 10: system msg for write_line failure : Bad file descriptor > 10: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 10: MPIR_Init_thread(152).......: > 10: MPID_Init(597)..............: > 10: MPIDI_UCX_mpi_init_hook(247): > 10: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 10: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 10: : > 10: system msg for write_line failure : Bad file descriptor > 10: [ip-172-31-13-129:15223:0:15223] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 10: ==== backtrace (tid: 15223) ==== > 10: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f830e911ea4] > 10: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f830e9120af] > 10: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f830e91226a] > 10: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f830f916140] > 10: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f830f149431] > 10: 5 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0xc8da5) [0x55c3c6b6ada5] > 10: 6 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x842c9) [0x55c3c6b262c9] > 10: 7 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x58c40) [0x55c3c6afac40] > 10: 8 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x3cf3d) [0x55c3c6adef3d] > 10: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f830e964d0a] > 10: 10 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x3d87a) [0x55c3c6adf87a] > 10: ================================= > 10/30 Test #10: FFTUnitTests .....................***Exception: SegFault > 0.02 sec > [1611165257.405339] [ip-172-31-13-129:15223:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.405363] [ip-172-31-13-129:15223:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15223:0:15223] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15223) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f830e911ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f830e9120af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f830e91226a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f830f916140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f830f149431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0xc8da5) [0x55c3c6b6ada5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x842c9) [0x55c3c6b262c9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x58c40) [0x55c3c6afac40] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x3cf3d) [0x55c3c6adef3d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f830e964d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/fft-test(+0x3d87a) [0x55c3c6adf87a] > ================================= > > test 11 > Start 11: GpuUtilsUnitTests > > 11: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/GpuUtilsUnitTests.xml" > 11: Test timeout computed to be: 30 > 11: [1611165257.426336] [ip-172-31-13-129:15231:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 11: [1611165257.426358] [ip-172-31-13-129:15231:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 11: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 11: MPIR_Init_thread(152).......: > 11: MPID_Init(597)..............: > 11: MPIDI_UCX_mpi_init_hook(247): > 11: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 11: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 11: : > 11: system msg for write_line failure : Bad file descriptor > 11: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 11: MPIR_Init_thread(152).......: > 11: MPID_Init(597)..............: > 11: MPIDI_UCX_mpi_init_hook(247): > 11: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 11: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 11: : > 11: system msg for write_line failure : Bad file descriptor > 11: [ip-172-31-13-129:15231:0:15231] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 11: ==== backtrace (tid: 15231) ==== > 11: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f57b6af2ea4] > 11: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f57b6af30af] > 11: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f57b6af326a] > 11: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f57b7af1140] > 11: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f57b7324431] > 11: 5 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x104d45) > [0x55eb6ef06d45] > 11: 6 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0xc0c29) > [0x55eb6eec2c29] > 11: 7 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x83490) > [0x55eb6ee85490] > 11: 8 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x4088d) > [0x55eb6ee4288d] > 11: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f57b6b3fd0a] > 11: 10 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x4118a) > [0x55eb6ee4318a] > 11: ================================= > 11/30 Test #11: GpuUtilsUnitTests ................***Exception: SegFault > 0.02 sec > [1611165257.426336] [ip-172-31-13-129:15231:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.426358] [ip-172-31-13-129:15231:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15231:0:15231] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15231) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f57b6af2ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f57b6af30af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f57b6af326a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f57b7af1140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f57b7324431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x104d45) > [0x55eb6ef06d45] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0xc0c29) [0x55eb6eec2c29] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x83490) [0x55eb6ee85490] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x4088d) [0x55eb6ee4288d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f57b6b3fd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/gpu_utils-test(+0x4118a) [0x55eb6ee4318a] > ================================= > > test 12 > Start 12: HardwareUnitTests > > 12: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/HardwareUnitTests.xml" > 12: Test timeout computed to be: 30 > 12: [1611165257.447686] [ip-172-31-13-129:15239:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 12: [1611165257.447706] [ip-172-31-13-129:15239:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 12: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 12: MPIR_Init_thread(152).......: > 12: MPID_Init(597)..............: > 12: MPIDI_UCX_mpi_init_hook(247): > 12: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 12: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 12: : > 12: system msg for write_line failure : Bad file descriptor > 12: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 12: MPIR_Init_thread(152).......: > 12: MPID_Init(597)..............: > 12: MPIDI_UCX_mpi_init_hook(247): > 12: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 12: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 12: : > 12: system msg for write_line failure : Bad file descriptor > 12: [ip-172-31-13-129:15239:0:15239] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 12: ==== backtrace (tid: 15239) ==== > 12: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fa6b1b64ea4] > 12: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fa6b1b650af] > 12: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fa6b1b6526a] > 12: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fa6b2b91140] > 12: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fa6b23c4431] > 12: 5 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0xcb9f5) > [0x561546dd99f5] > 12: 6 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x81889) > [0x561546d8f889] > 12: 7 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x43f20) > [0x561546d51f20] > 12: 8 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x3958d) > [0x561546d4758d] > 12: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fa6b1bdfd0a] > 12: 10 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x39e4a) > [0x561546d47e4a] > 12: ================================= > 12/30 Test #12: HardwareUnitTests ................***Exception: SegFault > 0.02 sec > [1611165257.447686] [ip-172-31-13-129:15239:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.447706] [ip-172-31-13-129:15239:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15239:0:15239] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15239) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fa6b1b64ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fa6b1b650af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fa6b1b6526a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fa6b2b91140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fa6b23c4431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0xcb9f5) [0x561546dd99f5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x81889) [0x561546d8f889] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x43f20) [0x561546d51f20] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x3958d) [0x561546d4758d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fa6b1bdfd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/hardware-test(+0x39e4a) [0x561546d47e4a] > ================================= > > test 13 > Start 13: MathUnitTests > > 13: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/math-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/MathUnitTests.xml" > 13: Test timeout computed to be: 30 > 13: [1611165257.468320] [ip-172-31-13-129:15247:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 13: [1611165257.468340] [ip-172-31-13-129:15247:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 13: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 13: MPIR_Init_thread(152).......: > 13: MPID_Init(597)..............: > 13: MPIDI_UCX_mpi_init_hook(247): > 13: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 13: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 13: : > 13: system msg for write_line failure : Bad file descriptor > 13: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 13: MPIR_Init_thread(152).......: > 13: MPID_Init(597)..............: > 13: MPIDI_UCX_mpi_init_hook(247): > 13: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 13: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 13: : > 13: system msg for write_line failure : Bad file descriptor > 13: [ip-172-31-13-129:15247:0:15247] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 13: ==== backtrace (tid: 15247) ==== > 13: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2ce59e7ea4] > 13: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2ce59e80af] > 13: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2ce59e826a] > 13: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2ce69e6140] > 13: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2ce6219431] > 13: 5 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x1b6fb5) [0x55f69e70efb5] > 13: 6 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x162f89) [0x55f69e6baf89] > 13: 7 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x1360a0) [0x55f69e68e0a0] > 13: 8 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x6571d) [0x55f69e5bd71d] > 13: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f2ce5a34d0a] > 13: 10 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x6600a) [0x55f69e5be00a] > 13: ================================= > 13/30 Test #13: MathUnitTests ....................***Exception: SegFault > 0.02 sec > [1611165257.468320] [ip-172-31-13-129:15247:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.468340] [ip-172-31-13-129:15247:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15247:0:15247] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15247) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2ce59e7ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2ce59e80af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2ce59e826a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2ce69e6140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2ce6219431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x1b6fb5) [0x55f69e70efb5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x162f89) [0x55f69e6baf89] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x1360a0) [0x55f69e68e0a0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x6571d) [0x55f69e5bd71d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f2ce5a34d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/math-test(+0x6600a) [0x55f69e5be00a] > ================================= > > test 14 > Start 14: MdrunUtilityUnitTests > > 14: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/MdrunUtilityUnitTests.xml" > 14: Test timeout computed to be: 30 > 14: [1611165257.489699] [ip-172-31-13-129:15255:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 14: [1611165257.489718] [ip-172-31-13-129:15255:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 14: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 14: MPIR_Init_thread(152).......: > 14: MPID_Init(597)..............: > 14: MPIDI_UCX_mpi_init_hook(247): > 14: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 14: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 14: : > 14: system msg for write_line failure : Bad file descriptor > 14: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 14: MPIR_Init_thread(152).......: > 14: MPID_Init(597)..............: > 14: MPIDI_UCX_mpi_init_hook(247): > 14: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 14: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 14: : > 14: system msg for write_line failure : Bad file descriptor > 14: [ip-172-31-13-129:15255:0:15255] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 14: ==== backtrace (tid: 15255) ==== > 14: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fe9fd1b8ea4] > 14: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fe9fd1b90af] > 14: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fe9fd1b926a] > 14: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fe9fe1e5140] > 14: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fe9fda18431] > 14: 5 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0xdecb5) > [0x557218e31cb5] > 14: 6 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x997c9) > [0x557218dec7c9] > 14: 7 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x5bba0) > [0x557218daeba0] > 14: 8 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x3dd1d) > [0x557218d90d1d] > 14: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fe9fd233d0a] > 14: 10 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x3e65a) > [0x557218d9165a] > 14: ================================= > 14/30 Test #14: MdrunUtilityUnitTests ............***Exception: SegFault > 0.02 sec > [1611165257.489699] [ip-172-31-13-129:15255:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.489718] [ip-172-31-13-129:15255:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15255:0:15255] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15255) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fe9fd1b8ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fe9fd1b90af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fe9fd1b926a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fe9fe1e5140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fe9fda18431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0xdecb5) > [0x557218e31cb5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x997c9) > [0x557218dec7c9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x5bba0) > [0x557218daeba0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x3dd1d) > [0x557218d90d1d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fe9fd233d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-test(+0x3e65a) > [0x557218d9165a] > ================================= > > test 15 > Start 15: MdrunUtilityMpiUnitTests > > 15: Test command: /usr/bin/mpiexec.mpich "-np" "4" "-host" "localhost" > "/<<PKGBUILDDIR>>/build/mpich/bin/mdrunutility-mpi-test" > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/MdrunUtilityMpiUnitTests.xml" > 15: Test timeout computed to be: 30 > 15: [1611165257.527219] [ip-172-31-13-129:15268:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 15: [1611165257.527246] [ip-172-31-13-129:15268:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 15: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 15: MPIR_Init_thread(152).......: > 15: MPID_Init(597)..............: > 15: MPIDI_UCX_mpi_init_hook(247): > 15: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 15/30 Test #15: MdrunUtilityMpiUnitTests .........***Failed 0.04 sec > [1611165257.527219] [ip-172-31-13-129:15268:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.527246] [ip-172-31-13-129:15268:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > > test 16 > Start 16: MDSpanTests > > 16: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/MDSpanTests.xml" > 16: Test timeout computed to be: 30 > 16: [1611165257.550247] [ip-172-31-13-129:15297:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 16: [1611165257.550274] [ip-172-31-13-129:15297:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 16: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 16: MPIR_Init_thread(152).......: > 16: MPID_Init(597)..............: > 16: MPIDI_UCX_mpi_init_hook(247): > 16: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 16: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 16: : > 16: system msg for write_line failure : Bad file descriptor > 16: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 16: MPIR_Init_thread(152).......: > 16: MPID_Init(597)..............: > 16: MPIDI_UCX_mpi_init_hook(247): > 16: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 16: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 16: : > 16: system msg for write_line failure : Bad file descriptor > 16: [ip-172-31-13-129:15297:0:15297] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 16: ==== backtrace (tid: 15297) ==== > 16: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fa0f85a1ea4] > 16: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fa0f85a20af] > 16: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fa0f85a226a] > 16: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fa0f95a0140] > 16: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fa0f8dd3431] > 16: 5 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0xe16e5) > [0x557ff0ab06e5] > 16: 6 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x9deb9) > [0x557ff0a6ceb9] > 16: 7 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x60540) > [0x557ff0a2f540] > 16: 8 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x3f3bd) > [0x557ff0a0e3bd] > 16: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fa0f85eed0a] > 16: 10 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x3fc7a) > [0x557ff0a0ec7a] > 16: ================================= > 16/30 Test #16: MDSpanTests ......................***Exception: SegFault > 0.02 sec > [1611165257.550247] [ip-172-31-13-129:15297:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.550274] [ip-172-31-13-129:15297:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15297:0:15297] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15297) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fa0f85a1ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fa0f85a20af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fa0f85a226a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fa0f95a0140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fa0f8dd3431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0xe16e5) [0x557ff0ab06e5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x9deb9) [0x557ff0a6ceb9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x60540) [0x557ff0a2f540] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x3f3bd) [0x557ff0a0e3bd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fa0f85eed0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/mdspan-test(+0x3fc7a) [0x557ff0a0ec7a] > ================================= > > test 17 > Start 17: OnlineHelpUnitTests > > 17: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/OnlineHelpUnitTests.xml" > 17: Test timeout computed to be: 30 > 17: [1611165257.572739] [ip-172-31-13-129:15305:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 17: [1611165257.572762] [ip-172-31-13-129:15305:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 17: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 17: MPIR_Init_thread(152).......: > 17: MPID_Init(597)..............: > 17: MPIDI_UCX_mpi_init_hook(247): > 17: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 17: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 17: : > 17: system msg for write_line failure : Bad file descriptor > 17: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 17: MPIR_Init_thread(152).......: > 17: MPID_Init(597)..............: > 17: MPIDI_UCX_mpi_init_hook(247): > 17: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 17: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 17: : > 17: system msg for write_line failure : Bad file descriptor > 17: [ip-172-31-13-129:15305:0:15305] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 17: ==== backtrace (tid: 15305) ==== > 17: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2b66757ea4] > 17: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2b667580af] > 17: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2b6675826a] > 17: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2b67756140] > 17: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2b66f89431] > 17: 5 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0xd3ed5) > [0x56531bd4ced5] > 17: 6 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x8b639) > [0x56531bd04639] > 17: 7 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x4d2d0) > [0x56531bcc62d0] > 17: 8 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x3ef1d) > [0x56531bcb7f1d] > 17: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f2b667a4d0a] > 17: 10 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x3f87a) > [0x56531bcb887a] > 17: ================================= > 17/30 Test #17: OnlineHelpUnitTests ..............***Exception: SegFault > 0.02 sec > [1611165257.572739] [ip-172-31-13-129:15305:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.572762] [ip-172-31-13-129:15305:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15305:0:15305] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15305) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2b66757ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2b667580af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2b6675826a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2b67756140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2b66f89431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0xd3ed5) > [0x56531bd4ced5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x8b639) > [0x56531bd04639] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x4d2d0) > [0x56531bcc62d0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x3ef1d) > [0x56531bcb7f1d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f2b667a4d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/onlinehelp-test(+0x3f87a) > [0x56531bcb887a] > ================================= > > test 18 > Start 18: OptionsUnitTests > > 18: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/options-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/OptionsUnitTests.xml" > 18: Test timeout computed to be: 30 > 18: [1611165257.594631] [ip-172-31-13-129:15313:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 18: [1611165257.594654] [ip-172-31-13-129:15313:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 18: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 18: MPIR_Init_thread(152).......: > 18: MPID_Init(597)..............: > 18: MPIDI_UCX_mpi_init_hook(247): > 18: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 18: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 18: : > 18: system msg for write_line failure : Bad file descriptor > 18: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 18: MPIR_Init_thread(152).......: > 18: MPID_Init(597)..............: > 18: MPIDI_UCX_mpi_init_hook(247): > 18: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 18: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 18: : > 18: system msg for write_line failure : Bad file descriptor > 18: [ip-172-31-13-129:15313:0:15313] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 18: ==== backtrace (tid: 15313) ==== > 18: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2671e88ea4] > 18: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2671e890af] > 18: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2671e8926a] > 18: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2672e87140] > 18: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f26726ba431] > 18: 5 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x176545) > [0x559f7ad5e545] > 18: 6 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x12f039) > [0x559f7ad17039] > 18: 7 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0xf7b10) > [0x559f7acdfb10] > 18: 8 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x7262d) > [0x559f7ac5a62d] > 18: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f2671ed5d0a] > 18: 10 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x7306a) > [0x559f7ac5b06a] > 18: ================================= > 18/30 Test #18: OptionsUnitTests .................***Exception: SegFault > 0.02 sec > [1611165257.594631] [ip-172-31-13-129:15313:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.594654] [ip-172-31-13-129:15313:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15313:0:15313] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15313) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2671e88ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2671e890af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2671e8926a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2672e87140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f26726ba431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x176545) [0x559f7ad5e545] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x12f039) [0x559f7ad17039] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0xf7b10) [0x559f7acdfb10] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x7262d) [0x559f7ac5a62d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f2671ed5d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/options-test(+0x7306a) [0x559f7ac5b06a] > ================================= > > test 19 > Start 19: PbcutilUnitTest > > 19: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/PbcutilUnitTest.xml" > 19: Test timeout computed to be: 30 > 19: [1611165257.617187] [ip-172-31-13-129:15321:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 19: [1611165257.617210] [ip-172-31-13-129:15321:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 19: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 19: MPIR_Init_thread(152).......: > 19: MPID_Init(597)..............: > 19: MPIDI_UCX_mpi_init_hook(247): > 19: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 19: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 19: : > 19: system msg for write_line failure : Bad file descriptor > 19: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 19: MPIR_Init_thread(152).......: > 19: MPID_Init(597)..............: > 19: MPIDI_UCX_mpi_init_hook(247): > 19: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 19: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 19: : > 19: system msg for write_line failure : Bad file descriptor > 19: [ip-172-31-13-129:15321:0:15321] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 19: ==== backtrace (tid: 15321) ==== > 19: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fc196b58ea4] > 19: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fc196b590af] > 19: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fc196b5926a] > 19: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fc197b5d140] > 19: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fc197390431] > 19: 5 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0xc36d5) > [0x5639909856d5] > 19: 6 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x78f79) > [0x56399093af79] > 19: 7 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x4d8f0) > [0x56399090f8f0] > 19: 8 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x39ccd) > [0x5639908fbccd] > 19: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fc196babd0a] > 19: 10 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x3a58a) > [0x5639908fc58a] > 19: ================================= > 19/30 Test #19: PbcutilUnitTest ..................***Exception: SegFault > 0.02 sec > [1611165257.617187] [ip-172-31-13-129:15321:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.617210] [ip-172-31-13-129:15321:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15321:0:15321] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15321) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fc196b58ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fc196b590af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fc196b5926a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fc197b5d140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fc197390431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0xc36d5) [0x5639909856d5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x78f79) [0x56399093af79] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x4d8f0) [0x56399090f8f0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x39ccd) [0x5639908fbccd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fc196babd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/pbcutil-test(+0x3a58a) [0x5639908fc58a] > ================================= > > test 20 > Start 20: RandomUnitTests > > 20: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/random-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/RandomUnitTests.xml" > 20: Test timeout computed to be: 30 > 20: [1611165257.643414] [ip-172-31-13-129:15329:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 20: [1611165257.643438] [ip-172-31-13-129:15329:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 20: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 20: MPIR_Init_thread(152).......: > 20: MPID_Init(597)..............: > 20: MPIDI_UCX_mpi_init_hook(247): > 20: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 20: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 20: : > 20: system msg for write_line failure : Bad file descriptor > 20: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 20: MPIR_Init_thread(152).......: > 20: MPID_Init(597)..............: > 20: MPIDI_UCX_mpi_init_hook(247): > 20: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 20: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 20: : > 20: system msg for write_line failure : Bad file descriptor > 20: [ip-172-31-13-129:15329:0:15329] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 20: ==== backtrace (tid: 15329) ==== > 20: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2c5af51ea4] > 20: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2c5af520af] > 20: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2c5af5226a] > 20: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2c5bf50140] > 20: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2c5b783431] > 20: 5 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0xf7e95) > [0x55e38ef17e95] > 20: 6 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0xb3e99) > [0x55e38eed3e99] > 20: 7 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0x88ab0) > [0x55e38eea8ab0] > 20: 8 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0x4625d) > [0x55e38ee6625d] > 20: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f2c5af9ed0a] > 20: 10 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0x46e9a) > [0x55e38ee66e9a] > 20: ================================= > 20/30 Test #20: RandomUnitTests ..................***Exception: SegFault > 0.03 sec > [1611165257.643414] [ip-172-31-13-129:15329:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.643438] [ip-172-31-13-129:15329:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15329:0:15329] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15329) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f2c5af51ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f2c5af520af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f2c5af5226a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f2c5bf50140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2c5b783431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0xf7e95) [0x55e38ef17e95] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0xb3e99) [0x55e38eed3e99] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0x88ab0) [0x55e38eea8ab0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0x4625d) [0x55e38ee6625d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f2c5af9ed0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/random-test(+0x46e9a) [0x55e38ee66e9a] > ================================= > > test 21 > Start 21: RestraintTests > > 21: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/RestraintTests.xml" > 21: Test timeout computed to be: 30 > 21: [1611165257.665150] [ip-172-31-13-129:15337:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 21: [1611165257.665175] [ip-172-31-13-129:15337:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 21: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 21: MPIR_Init_thread(152).......: > 21: MPID_Init(597)..............: > 21: MPIDI_UCX_mpi_init_hook(247): > 21: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 21: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 21: : > 21: system msg for write_line failure : Bad file descriptor > 21: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 21: MPIR_Init_thread(152).......: > 21: MPID_Init(597)..............: > 21: MPIDI_UCX_mpi_init_hook(247): > 21: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 21: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 21: : > 21: system msg for write_line failure : Bad file descriptor > 21: [ip-172-31-13-129:15337:0:15337] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 21: ==== backtrace (tid: 15337) ==== > 21: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f226c74bea4] > 21: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f226c74c0af] > 21: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f226c74c26a] > 21: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f226d74a140] > 21: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f226cf7d431] > 21: 5 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0xbe6a5) > [0x55913c0be6a5] > 21: 6 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x782a9) > [0x55913c0782a9] > 21: 7 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x3a9c0) > [0x55913c03a9c0] > 21: 8 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x3875d) > [0x55913c03875d] > 21: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f226c798d0a] > 21: 10 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x3904a) > [0x55913c03904a] > 21: ================================= > 21/30 Test #21: RestraintTests ...................***Exception: SegFault > 0.02 sec > [1611165257.665150] [ip-172-31-13-129:15337:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.665175] [ip-172-31-13-129:15337:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15337:0:15337] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15337) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f226c74bea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f226c74c0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f226c74c26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f226d74a140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f226cf7d431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0xbe6a5) > [0x55913c0be6a5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x782a9) > [0x55913c0782a9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x3a9c0) > [0x55913c03a9c0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x3875d) > [0x55913c03875d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f226c798d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/restraintpotential-test(+0x3904a) > [0x55913c03904a] > ================================= > > test 22 > Start 22: TableUnitTests > > 22: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/table-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/TableUnitTests.xml" > 22: Test timeout computed to be: 30 > 22: [1611165257.686156] [ip-172-31-13-129:15345:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 22: [1611165257.686178] [ip-172-31-13-129:15345:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 22: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 22: MPIR_Init_thread(152).......: > 22: MPID_Init(597)..............: > 22: MPIDI_UCX_mpi_init_hook(247): > 22: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 22: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 22: : > 22: system msg for write_line failure : Bad file descriptor > 22: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 22: MPIR_Init_thread(152).......: > 22: MPID_Init(597)..............: > 22: MPIDI_UCX_mpi_init_hook(247): > 22: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 22: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 22: : > 22: system msg for write_line failure : Bad file descriptor > 22: [ip-172-31-13-129:15345:0:15345] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 22: ==== backtrace (tid: 15345) ==== > 22: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7efdeac86ea4] > 22: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7efdeac870af] > 22: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7efdeac8726a] > 22: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7efdebc85140] > 22: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7efdeb4b8431] > 22: 5 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0xf6a15) [0x55b0b7bc5a15] > 22: 6 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0xac999) [0x55b0b7b7b999] > 22: 7 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0x6fd70) [0x55b0b7b3ed70] > 22: 8 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0x421bd) [0x55b0b7b111bd] > 22: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7efdeacd3d0a] > 22: 10 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0x42a7a) [0x55b0b7b11a7a] > 22: ================================= > 22/30 Test #22: TableUnitTests ...................***Exception: SegFault > 0.02 sec > [1611165257.686156] [ip-172-31-13-129:15345:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.686178] [ip-172-31-13-129:15345:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15345:0:15345] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15345) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7efdeac86ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7efdeac870af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7efdeac8726a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7efdebc85140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7efdeb4b8431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0xf6a15) [0x55b0b7bc5a15] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0xac999) [0x55b0b7b7b999] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0x6fd70) [0x55b0b7b3ed70] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0x421bd) [0x55b0b7b111bd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7efdeacd3d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/table-test(+0x42a7a) [0x55b0b7b11a7a] > ================================= > > test 23 > Start 23: TaskAssignmentUnitTests > > 23: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/TaskAssignmentUnitTests.xml" > 23: Test timeout computed to be: 30 > 23: [1611165257.706854] [ip-172-31-13-129:15353:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 23: [1611165257.706874] [ip-172-31-13-129:15353:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 23: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 23: MPIR_Init_thread(152).......: > 23: MPID_Init(597)..............: > 23: MPIDI_UCX_mpi_init_hook(247): > 23: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 23: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 23: : > 23: system msg for write_line failure : Bad file descriptor > 23: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 23: MPIR_Init_thread(152).......: > 23: MPID_Init(597)..............: > 23: MPIDI_UCX_mpi_init_hook(247): > 23: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 23: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 23: : > 23: system msg for write_line failure : Bad file descriptor > 23: [ip-172-31-13-129:15353:0:15353] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 23: ==== backtrace (tid: 15353) ==== > 23: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fef56631ea4] > 23: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fef566320af] > 23: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fef5663226a] > 23: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fef57630140] > 23: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fef56e63431] > 23: 5 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0xcfe65) > [0x56364ea4de65] > 23: 6 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x8a449) > [0x56364ea08449] > 23: 7 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x4cdb0) > [0x56364e9cadb0] > 23: 8 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x3b3dd) > [0x56364e9b93dd] > 23: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fef5667ed0a] > 23: 10 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x3bc9a) > [0x56364e9b9c9a] > 23: ================================= > 23/30 Test #23: TaskAssignmentUnitTests ..........***Exception: SegFault > 0.02 sec > [1611165257.706854] [ip-172-31-13-129:15353:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.706874] [ip-172-31-13-129:15353:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15353:0:15353] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15353) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fef56631ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fef566320af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fef5663226a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fef57630140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fef56e63431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0xcfe65) > [0x56364ea4de65] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x8a449) > [0x56364ea08449] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x4cdb0) > [0x56364e9cadb0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x3b3dd) > [0x56364e9b93dd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fef5667ed0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/taskassignment-test(+0x3bc9a) > [0x56364e9b9c9a] > ================================= > > test 24 > Start 24: TopologyTest > > 24: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/topology-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/TopologyTest.xml" > 24: Test timeout computed to be: 30 > 24: [1611165257.727790] [ip-172-31-13-129:15361:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 24: [1611165257.727813] [ip-172-31-13-129:15361:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 24: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 24: MPIR_Init_thread(152).......: > 24: MPID_Init(597)..............: > 24: MPIDI_UCX_mpi_init_hook(247): > 24: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 24: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 24: : > 24: system msg for write_line failure : Bad file descriptor > 24: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 24: MPIR_Init_thread(152).......: > 24: MPID_Init(597)..............: > 24: MPIDI_UCX_mpi_init_hook(247): > 24: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 24: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 24: : > 24: system msg for write_line failure : Bad file descriptor > 24: [ip-172-31-13-129:15361:0:15361] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 24: ==== backtrace (tid: 15361) ==== > 24: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f0c0484bea4] > 24: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f0c0484c0af] > 24: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f0c0484c26a] > 24: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f0c0584a140] > 24: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f0c0507d431] > 24: 5 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0xe0955) > [0x55bdcea51955] > 24: 6 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x8d959) > [0x55bdce9fe959] > 24: 7 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x622d0) > [0x55bdce9d32d0] > 24: 8 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x3eaed) > [0x55bdce9afaed] > 24: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f0c04898d0a] > 24: 10 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x3f3fa) > [0x55bdce9b03fa] > 24: ================================= > 24/30 Test #24: TopologyTest .....................***Exception: SegFault > 0.02 sec > [1611165257.727790] [ip-172-31-13-129:15361:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.727813] [ip-172-31-13-129:15361:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15361:0:15361] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15361) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f0c0484bea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f0c0484c0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f0c0484c26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f0c0584a140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f0c0507d431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0xe0955) [0x55bdcea51955] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x8d959) [0x55bdce9fe959] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x622d0) [0x55bdce9d32d0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x3eaed) [0x55bdce9afaed] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f0c04898d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/topology-test(+0x3f3fa) [0x55bdce9b03fa] > ================================= > > test 25 > Start 25: PullTest > > 25: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/pull-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/PullTest.xml" > 25: Test timeout computed to be: 30 > 25: [1611165257.748422] [ip-172-31-13-129:15369:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 25: [1611165257.748447] [ip-172-31-13-129:15369:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 25: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 25: MPIR_Init_thread(152).......: > 25: MPID_Init(597)..............: > 25: MPIDI_UCX_mpi_init_hook(247): > 25: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 25: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 25: : > 25: system msg for write_line failure : Bad file descriptor > 25: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 25: MPIR_Init_thread(152).......: > 25: MPID_Init(597)..............: > 25: MPIDI_UCX_mpi_init_hook(247): > 25: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 25: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 25: : > 25: system msg for write_line failure : Bad file descriptor > 25: [ip-172-31-13-129:15369:0:15369] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 25: ==== backtrace (tid: 15369) ==== > 25: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f6c5163cea4] > 25: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f6c5163d0af] > 25: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f6c5163d26a] > 25: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f6c52641140] > 25: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f6c51e74431] > 25: 5 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0xe5b35) [0x556d7e1afb35] > 25: 6 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x80eb9) [0x556d7e14aeb9] > 25: 7 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x44170) [0x556d7e10e170] > 25: 8 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x3f80d) [0x556d7e10980d] > 25: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f6c5168fd0a] > 25: 10 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x4019a) [0x556d7e10a19a] > 25: ================================= > 25/30 Test #25: PullTest .........................***Exception: SegFault > 0.02 sec > [1611165257.748422] [ip-172-31-13-129:15369:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.748447] [ip-172-31-13-129:15369:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15369:0:15369] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15369) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f6c5163cea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f6c5163d0af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f6c5163d26a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f6c52641140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f6c51e74431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0xe5b35) [0x556d7e1afb35] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x80eb9) [0x556d7e14aeb9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x44170) [0x556d7e10e170] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x3f80d) [0x556d7e10980d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f6c5168fd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/pull-test(+0x4019a) [0x556d7e10a19a] > ================================= > > test 26 > Start 26: AwhTest > > 26: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/awh-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/AwhTest.xml" > 26: Test timeout computed to be: 30 > 26: [1611165257.769432] [ip-172-31-13-129:15377:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 26: [1611165257.769458] [ip-172-31-13-129:15377:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 26: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 26: MPIR_Init_thread(152).......: > 26: MPID_Init(597)..............: > 26: MPIDI_UCX_mpi_init_hook(247): > 26: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 26: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 26: : > 26: system msg for write_line failure : Bad file descriptor > 26: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 26: MPIR_Init_thread(152).......: > 26: MPID_Init(597)..............: > 26: MPIDI_UCX_mpi_init_hook(247): > 26: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 26: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 26: : > 26: system msg for write_line failure : Bad file descriptor > 26: [ip-172-31-13-129:15377:0:15377] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 26: ==== backtrace (tid: 15377) ==== > 26: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f1b33140ea4] > 26: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f1b331410af] > 26: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f1b3314126a] > 26: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f1b3413f140] > 26: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f1b33972431] > 26: 5 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0xfe585) [0x562f455a7585] > 26: 6 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x97229) [0x562f45540229] > 26: 7 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x6b980) [0x562f45514980] > 26: 8 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x3e81d) [0x562f454e781d] > 26: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f1b3318dd0a] > 26: 10 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x3f30a) [0x562f454e830a] > 26: ================================= > 26/30 Test #26: AwhTest ..........................***Exception: SegFault > 0.02 sec > [1611165257.769432] [ip-172-31-13-129:15377:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.769458] [ip-172-31-13-129:15377:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15377:0:15377] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15377) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f1b33140ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f1b331410af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f1b3314126a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f1b3413f140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f1b33972431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0xfe585) [0x562f455a7585] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x97229) [0x562f45540229] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x6b980) [0x562f45514980] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x3e81d) [0x562f454e781d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f1b3318dd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/awh-test(+0x3f30a) [0x562f454e830a] > ================================= > > test 27 > Start 27: SimdUnitTests > > 27: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/simd-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/SimdUnitTests.xml" > 27: Test timeout computed to be: 30 > 27: [1611165257.791495] [ip-172-31-13-129:15385:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 27: [1611165257.791521] [ip-172-31-13-129:15385:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 27: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 27: MPIR_Init_thread(152).......: > 27: MPID_Init(597)..............: > 27: MPIDI_UCX_mpi_init_hook(247): > 27: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 27: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 27: : > 27: system msg for write_line failure : Bad file descriptor > 27: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 27: MPIR_Init_thread(152).......: > 27: MPID_Init(597)..............: > 27: MPIDI_UCX_mpi_init_hook(247): > 27: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 27: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 27: : > 27: system msg for write_line failure : Bad file descriptor > 27: [ip-172-31-13-129:15385:0:15385] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 27: ==== backtrace (tid: 15385) ==== > 27: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f36d1530ea4] > 27: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f36d15310af] > 27: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f36d153126a] > 27: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f36d252f140] > 27: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f36d1d62431] > 27: 5 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x16c385) [0x5636ac4fa385] > 27: 6 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x128ee9) [0x5636ac4b6ee9] > 27: 7 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0xfd640) [0x5636ac48b640] > 27: 8 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x749bd) [0x5636ac4029bd] > 27: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f36d157dd0a] > 27: 10 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x7527a) [0x5636ac40327a] > 27: ================================= > 27/30 Test #27: SimdUnitTests ....................***Exception: SegFault > 0.02 sec > [1611165257.791495] [ip-172-31-13-129:15385:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.791521] [ip-172-31-13-129:15385:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15385:0:15385] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15385) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f36d1530ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f36d15310af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f36d153126a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f36d252f140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f36d1d62431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x16c385) [0x5636ac4fa385] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x128ee9) [0x5636ac4b6ee9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0xfd640) [0x5636ac48b640] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x749bd) [0x5636ac4029bd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f36d157dd0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/simd-test(+0x7527a) [0x5636ac40327a] > ================================= > > test 28 > Start 28: CompatibilityHelpersTests > > 28: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/compat-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/CompatibilityHelpersTests.xml" > 28: Test timeout computed to be: 30 > 28: [1611165257.813687] [ip-172-31-13-129:15393:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 28: [1611165257.813713] [ip-172-31-13-129:15393:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 28: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 28: MPIR_Init_thread(152).......: > 28: MPID_Init(597)..............: > 28: MPIDI_UCX_mpi_init_hook(247): > 28: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 28: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 28: : > 28: system msg for write_line failure : Bad file descriptor > 28: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 28: MPIR_Init_thread(152).......: > 28: MPID_Init(597)..............: > 28: MPIDI_UCX_mpi_init_hook(247): > 28: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 28: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 28: : > 28: system msg for write_line failure : Bad file descriptor > 28: [ip-172-31-13-129:15393:0:15393] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 28: ==== backtrace (tid: 15393) ==== > 28: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fecb88a3ea4] > 28: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fecb88a40af] > 28: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fecb88a426a] > 28: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fecb98a2140] > 28: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fecb90d5431] > 28: 5 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x1178f5) > [0x55ea7d7f98f5] > 28: 6 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0xd40c9) > [0x55ea7d7b60c9] > 28: 7 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x968d0) > [0x55ea7d7788d0] > 28: 8 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x5cc1d) > [0x55ea7d73ec1d] > 28: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fecb88f0d0a] > 28: 10 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x5d4da) > [0x55ea7d73f4da] > 28: ================================= > 28/30 Test #28: CompatibilityHelpersTests ........***Exception: SegFault > 0.02 sec > [1611165257.813687] [ip-172-31-13-129:15393:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.813713] [ip-172-31-13-129:15393:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15393:0:15393] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15393) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fecb88a3ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fecb88a40af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fecb88a426a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fecb98a2140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fecb90d5431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x1178f5) [0x55ea7d7f98f5] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0xd40c9) [0x55ea7d7b60c9] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x968d0) [0x55ea7d7788d0] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x5cc1d) [0x55ea7d73ec1d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fecb88f0d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/compat-test(+0x5d4da) [0x55ea7d73f4da] > ================================= > > test 29 > Start 29: FileIOTests > > 29: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/FileIOTests.xml" > 29: Test timeout computed to be: 30 > 29: [1611165257.835741] [ip-172-31-13-129:15401:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 29: [1611165257.835763] [ip-172-31-13-129:15401:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 29: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 29: MPIR_Init_thread(152).......: > 29: MPID_Init(597)..............: > 29: MPIDI_UCX_mpi_init_hook(247): > 29: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 29: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 29: : > 29: system msg for write_line failure : Bad file descriptor > 29: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 29: MPIR_Init_thread(152).......: > 29: MPID_Init(597)..............: > 29: MPIDI_UCX_mpi_init_hook(247): > 29: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 29: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 29: : > 29: system msg for write_line failure : Bad file descriptor > 29: [ip-172-31-13-129:15401:0:15401] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 29: ==== backtrace (tid: 15401) ==== > 29: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fbfa4fa2ea4] > 29: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fbfa4fa30af] > 29: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fbfa4fa326a] > 29: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fbfa5fa7140] > 29: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fbfa57da431] > 29: 5 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x179385) > [0x55c7e60b8385] > 29: 6 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0xb9999) > [0x55c7e5ff8999] > 29: 7 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x87900) > [0x55c7e5fc6900] > 29: 8 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x524cd) > [0x55c7e5f914cd] > 29: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7fbfa4ff5d0a] > 29: 10 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x531ca) > [0x55c7e5f921ca] > 29: ================================= > 29/30 Test #29: FileIOTests ......................***Exception: SegFault > 0.02 sec > [1611165257.835741] [ip-172-31-13-129:15401:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.835763] [ip-172-31-13-129:15401:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15401:0:15401] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15401) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7fbfa4fa2ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7fbfa4fa30af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7fbfa4fa326a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fbfa5fa7140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7fbfa57da431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x179385) [0x55c7e60b8385] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0xb9999) [0x55c7e5ff8999] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x87900) [0x55c7e5fc6900] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x524cd) [0x55c7e5f914cd] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fbfa4ff5d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/fileio-test(+0x531ca) [0x55c7e5f921ca] > ================================= > > test 30 > Start 30: SelectionUnitTests > > 30: Test command: /<<PKGBUILDDIR>>/build/mpich/bin/selection-test > "--gtest_output=xml:/<<PKGBUILDDIR>>/build/mpich/Testing/Temporary/SelectionUnitTests.xml" > 30: Test timeout computed to be: 30 > 30: [1611165257.857831] [ip-172-31-13-129:15409:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > 30: [1611165257.857854] [ip-172-31-13-129:15409:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > 30: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 30: MPIR_Init_thread(152).......: > 30: MPID_Init(597)..............: > 30: MPIDI_UCX_mpi_init_hook(247): > 30: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 30: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 30: : > 30: system msg for write_line failure : Bad file descriptor > 30: Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in > PMPI_Init_thread: Other MPI error, error stack: > 30: MPIR_Init_thread(152).......: > 30: MPID_Init(597)..............: > 30: MPIDI_UCX_mpi_init_hook(247): > 30: init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > 30: [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > 30: : > 30: system msg for write_line failure : Bad file descriptor > 30: [ip-172-31-13-129:15409:0:15409] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > 30: ==== backtrace (tid: 15409) ==== > 30: 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f28302c5ea4] > 30: 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f28302c60af] > 30: 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f28302c626a] > 30: 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f28312ca140] > 30: 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2830afd431] > 30: 5 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0x21a835) > [0x55af792a7835] > 30: 6 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0x15b179) > [0x55af791e8179] > 30: 7 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0x12f060) > [0x55af791bc060] > 30: 8 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0xa075d) > [0x55af7912d75d] > 30: 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) > [0x7f2830318d0a] > 30: 10 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0xa145a) > [0x55af7912e45a] > 30: ================================= > 30/30 Test #30: SelectionUnitTests ...............***Exception: SegFault > 0.02 sec > [1611165257.857831] [ip-172-31-13-129:15409:0] rdmacm_cm.c:638 UCX > ERROR rdma_create_event_channel failed: No such device > [1611165257.857854] [ip-172-31-13-129:15409:0] ucp_worker.c:1432 UCX > ERROR failed to open CM on component rdmacm with status Input/output error > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > Abort(1615247) on node 0 (rank 0 in comm 0): Fatal error in PMPI_Init_thread: > Other MPI error, error stack: > MPIR_Init_thread(152).......: > MPID_Init(597)..............: > MPIDI_UCX_mpi_init_hook(247): > init_worker(71).............: ucx function returned with failed > status(ucx_init.c 71 init_worker Input/output error) > [unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1615247 > : > system msg for write_line failure : Bad file descriptor > [ip-172-31-13-129:15409:0:15409] Caught signal 11 (Segmentation fault: > address not mapped to object at address (nil)) > ==== backtrace (tid: 15409) ==== > 0 /usr/lib/x86_64-linux-gnu/libucs.so.0(ucs_handle_error+0x2a4) > [0x7f28302c5ea4] > 1 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x220af) [0x7f28302c60af] > 2 /usr/lib/x86_64-linux-gnu/libucs.so.0(+0x2226a) [0x7f28302c626a] > 3 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7f28312ca140] > 4 /usr/lib/x86_64-linux-gnu/libmpich.so.12(MPIR_Err_return_comm+0xa1) > [0x7f2830afd431] > 5 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0x21a835) > [0x55af792a7835] > 6 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0x15b179) > [0x55af791e8179] > 7 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0x12f060) > [0x55af791bc060] > 8 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0xa075d) [0x55af7912d75d] > 9 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f2830318d0a] > 10 /<<PKGBUILDDIR>>/build/mpich/bin/selection-test(+0xa145a) [0x55af7912e45a] > ================================= > > > 0% tests passed, 30 tests failed out of 30 > > Label Time Summary: > GTest = 0.77 sec*proc (30 tests) > MpiTest = 0.11 sec*proc (3 tests) > UnitTest = 0.77 sec*proc (30 tests) > > Total Test time (real) = 0.81 sec > > The following tests FAILED: > 1 - TestUtilsUnitTests (SEGFAULT) > 2 - TestUtilsMpiUnitTests (Failed) > 3 - UtilityUnitTests (SEGFAULT) > 4 - UtilityMpiUnitTests (Failed) > 5 - MdlibUnitTest (SEGFAULT) > 6 - AppliedForcesUnitTest (SEGFAULT) > 7 - CommandLineUnitTests (SEGFAULT) > 8 - DomDecTests (SEGFAULT) > 9 - EwaldUnitTests (SEGFAULT) > 10 - FFTUnitTests (SEGFAULT) > 11 - GpuUtilsUnitTests (SEGFAULT) > 12 - HardwareUnitTests (SEGFAULT) > 13 - MathUnitTests (SEGFAULT) > 14 - MdrunUtilityUnitTests (SEGFAULT) > 15 - MdrunUtilityMpiUnitTests (Failed) > 16 - MDSpanTests (SEGFAULT) > 17 - OnlineHelpUnitTests (SEGFAULT) > 18 - OptionsUnitTests (SEGFAULT) > 19 - PbcutilUnitTest (SEGFAULT) > 20 - RandomUnitTests (SEGFAULT) > 21 - RestraintTests (SEGFAULT) > 22 - TableUnitTests (SEGFAULT) > 23 - TaskAssignmentUnitTests (SEGFAULT) > 24 - TopologyTest (SEGFAULT) > 25 - PullTest (SEGFAULT) > 26 - AwhTest (SEGFAULT) > 27 - SimdUnitTests (SEGFAULT) > 28 - CompatibilityHelpersTests (SEGFAULT) > 29 - FileIOTests (SEGFAULT) > 30 - SelectionUnitTests (SEGFAULT) > Errors while running CTest > make: *** [debian/rules:158: build-mpich] Error 1 The full build log is available from: http://qa-logs.debian.net/2021/01/20/gromacs_2020.4-2_unstable.log A list of current common problems and possible solutions is available at http://wiki.debian.org/qa.debian.org/FTBFS . You're welcome to contribute! If you reassign this bug to another package, please marking it as 'affects'-ing this package. See https://www.debian.org/Bugs/server-control#affects If you fail to reproduce this, please provide a build log and diff it with me so that we can identify if something relevant changed in the meantime. About the archive rebuild: The rebuild was done on EC2 VM instances from Amazon Web Services, using a clean, minimal and up-to-date chroot. Every failed build was retried once to eliminate random failures.
