On Sun, Jun 12, 2022 at 5:38 PM Drew Parsons <[email protected]> wrote: > > Hi MIPS porters, I need helping with the build of fenics-dolfinx on > mipsel from experimental, > https://salsa.debian.org/science-team/fenics/fenics-dolfinx/-/tree/experimental > > It previously built fine, but the latest version 1:0.4.1 has evidently > grown large enough to break on mipsel. > > mip64el builds successfully, as we might expect. But it's not simply a > 32-bit issue in the sense that i386, armel, armhf are all still building > successfully. > > Build logs can be found at > https://buildd.debian.org/status/logs.php?pkg=fenics-dolfinx&arch=mipsel > > In the build for 1:0.4.1-1exp1 we see: > > [ 83%] Building CXX object CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o > /usr/bin/c++ -DBOOST_ALL_NO_LIB -DBOOST_CHRONO_DYN_LINK > -DBOOST_TIMER_DYN_LINK -DDOLFINX_VERSION=\"0.4.1\" -DHAS_PTSCOTCH > -DHAS_SLEPC -D_FILE_OFFSET_BITS=64 -Dcpp_EXPORTS > -I/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib/python3/dist-packages/petsc4py/include > -I/usr/lib/python3/dist-packages/mpi4py/include -isystem > /usr/include/python3.9 -isystem > /usr/lib/python3/dist-packages/pybind11/include -isystem > /usr/lib/python3/dist-packages/ffcx/codegeneration -isystem > /usr/lib/mipsel-linux-gnu/openmpi/include -isystem > /usr/lib/mipsel-linux-gnu/openmpi/include/openmpi -isystem > /usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/include -isystem > /usr/include/hdf5/openmpi -isystem > /usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/include -g -O2 > -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat > -Werror=format-security -isystem > /<<PKGBUILDDIR>>/debian/tmp-complex/usr/include -DVERSION_INFO=\"0.4.1\" > -O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Werror -pedantic -flto > -fno-fat-lto-objects -pthread -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++17 > -MD -MT CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o -MF > CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o.d -o > CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o -c > /<<PKGBUILDDIR>>/python/dolfinx/wrappers/nls.cpp > [ 91%] Building CXX object > CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o > /usr/bin/c++ -DBOOST_ALL_NO_LIB -DBOOST_CHRONO_DYN_LINK > -DBOOST_TIMER_DYN_LINK -DDOLFINX_VERSION=\"0.4.1\" -DHAS_PTSCOTCH > -DHAS_SLEPC -D_FILE_OFFSET_BITS=64 -Dcpp_EXPORTS > -I/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib/python3/dist-packages/petsc4py/include > -I/usr/lib/python3/dist-packages/mpi4py/include -isystem > /usr/include/python3.9 -isystem > /usr/lib/python3/dist-packages/pybind11/include -isystem > /usr/lib/python3/dist-packages/ffcx/codegeneration -isystem > /usr/lib/mipsel-linux-gnu/openmpi/include -isystem > /usr/lib/mipsel-linux-gnu/openmpi/include/openmpi -isystem > /usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/include -isystem > /usr/include/hdf5/openmpi -isystem > /usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/include -g -O2 > -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat > -Werror=format-security -isystem > /<<PKGBUILDDIR>>/debian/tmp-complex/usr/include -DVERSION_INFO=\"0.4.1\" > -O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Werror -pedantic -flto > -fno-fat-lto-objects -pthread -Wdate-time -D_FORTIFY_SOURCE=2 -std=c++17 > -MD -MT CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o -MF > CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o.d -o > CMakeFiles/cpp.dir/dolfinx/wrappers/refinement.cpp.o -c > /<<PKGBUILDDIR>>/python/dolfinx/wrappers/refinement.cpp > > cc1plus: out of memory allocating 17707045 bytes after a total of > 42041344 bytes > > > In the first instance I tried switching off LTO, adding the cmake > configuration "set(CMAKE_INTERPROCEDURAL_OPTIMIZATION FALSE)" which > deactivates -flto. > But it didn't fix the problem. The corresponding log for 1:0.4.1-1exp2 > shows much the same problem, > > virtual memory exhausted: Cannot allocate memory > > I then tried building on the porterbox eller.debian.org, reducing > optimization from -O3 to -O2, but that gave the same result. > > Finally I tried with no optimizations at all (-O0) via cmake option > add_compile_options(-O0). This did allow the object files to build, but > then linking failed with relocation truncated: > > [100%] Linking CXX shared module > ../lib.linux-mips-3.10/dolfinx/cpp.cpython-310-mipsel-linux-gnu.so > /usr/bin/cmake -E cmake_link_script CMakeFiles/cpp.dir/link.txt > --verbose=1 > /usr/bin/c++ -fPIC -g -O2 > -ffile-prefix-map=/home/dparsons/dolfinx/fenics-dolfinx=. > -fstack-protector-strong -Wformat -Werror=format-security -isystem > /home/dparsons/dolfinx/fenics-dolfinx/debian/tmp-complex/usr > /include -DVERSION_INFO=\"0.4.1\" -O3 -DNDEBUG -pthread -Wl,-z,relro > -shared -o > ../lib.linux-mips-3.10/dolfinx/cpp.cpython-310-mipsel-linux-gnu.so > CMakeFiles/cpp.dir/dolfinx/wrappers/dolfinx.cpp.o CMakeFiles/cp > p.dir/dolfinx/wrappers/common.cpp.o > CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o > CMakeFiles/cpp.dir/dolfinx/wrappers/geometry.cpp.o > CMakeFiles/cpp.dir/dolfinx/wrappers/graph.cpp.o > CMakeFiles/cpp.dir/dolfinx/wra > ppers/io.cpp.o CMakeFiles/cpp.dir/dolfinx/wrappers/la.cpp.o > CMakeFiles/cpp.dir/dolfinx/wrappers/log.cpp.o > CMakeFiles/cpp.dir/dolfinx/wrappers/mesh.cpp.o > CMakeFiles/cpp.dir/dolfinx/wrappers/nls.cpp.o CMakeFiles/c > pp.dir/dolfinx/wrappers/refinement.cpp.o > -Wl,-rpath,/home/dparsons/dolfinx/fenics-dolfinx/debian/tmp-complex/usr/lib/mipsel-linux-gnu:/usr/lib/mipsel-linux-gnu/openmpi/lib:/usr/lib/mipsel-linux-gnu/hdf5/openmpi > :/usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/lib:/usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib > /home/dparsons/dolfinx/fenics-dolfinx/debian/tmp-complex/usr/lib/mipsel-linux-gnu/libdolfinx_com > plex.so.0.4.1 /usr/lib/mipsel-linux-gnu/libbasix.so.0.4.2 > /usr/lib/mipsel-linux-gnu/libboost_timer.so.1.74.0 > /usr/lib/mipsel-linux-gnu/libboost_chrono.so.1.74.0 > /usr/lib/mipsel-linux-gnu/openmpi/lib/libmpi_cxx.so > /usr/lib/mipsel-linux-gnu/openmpi/lib/libmpi.so > /usr/lib/mipsel-linux-gnu/hdf5/openmpi/libhdf5.so > /usr/lib/slepcdir/slepc3.16/mipsel-linux-gnu-complex/lib/libslepc_complex.so > /usr/lib/petscdir/petsc3.16/mipsel-linux-gnu-complex/lib/libpetsc_complex.so > CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o: in function > `_PyObject_TypeCheck': > /usr/include/python3.10/object.h:247:(.text+0xa0): relocation truncated > to fit: R_MIPS_CALL16 against `PyType_IsSubtype' > CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o: in function `_Py_DECREF': > /usr/include/python3.10/object.h:500:(.text+0x174): relocation truncated > to fit: R_MIPS_CALL16 against `_Py_Dealloc' > CMakeFiles/cpp.dir/dolfinx/wrappers/fem.cpp.o: in function > `__Pyx_ImportFunction(_object*, char const*, void (**)(), char const*)': > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:111:(.text+0x46c): > relocation truncated to fit: R_MIPS_CALL16 against > `PyObject_GetAttrString' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:114:(.text+0x498): > relocation truncated to fit: R_MIPS_CALL16 against > `PyDict_GetItemString' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:116:(.text+0x4bc): > relocation truncated to fit: R_MIPS_GOT16 against `PyExc_ImportError' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:116:(.text+0x4c8): > relocation truncated to fit: R_MIPS_CALL16 against `PyModule_GetName' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:116:(.text+0x4f0): > relocation truncated to fit: R_MIPS_CALL16 against `PyErr_Format' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:122:(.text+0x514): > relocation truncated to fit: R_MIPS_CALL16 against `PyCapsule_IsValid' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:123:(.text+0x538): > relocation truncated to fit: R_MIPS_GOT16 against `PyExc_TypeError' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:123:(.text+0x544): > relocation truncated to fit: R_MIPS_CALL16 against `PyModule_GetName' > /usr/lib/python3/dist-packages/mpi4py/include/mpi4py/mpi4py.MPI_api.h:123:(.text+0x560): > additional relocation overflows omitted from the output > collect2: error: ld returned 1 exit status > gmake[4]: *** [CMakeFiles/cpp.dir/build.make:266: > ../lib.linux-mips-3.10/dolfinx/cpp.cpython-310-mipsel-linux-gnu.so] > Error 1 > > > At this point I'm out of clues. What do you recommend?
I used to see something similar when building Crypto++ on resource constrained dev-boards without swap files, like a BeagleBoard. cc1plus used to crash all the time when certain source files were built, especially ones that included a lot of header files. The symptom was the same as: > cc1plus: out of memory allocating 17707045 > bytes after a total of 42041344 bytes Drop back to 1 make job. Don't use `make -j N`. Instead, just use `make` or `make -j 1`. Jeff
