How big are these matrices? The memory PETSc allocates is (8+4)*num_nnz + 4*num_rows bytes (in double precision) https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/blob/main/src/mat/impls/aij/seq/aij.h*L47__;Iw!!G_uCfscf7eWS!eQLvMSRayHm9_oWVvaVFu0KZKnXMgFnQ06zPI0ciy6V4RLioHdXqliTOEFJQs8W8qb-y8v2-wVe3mKbGwBDucIVgolkrDHg$ Assuming num_nnz >> num_rows, 2.8 billion nonzeros fit 32GB of RAM.
Anyway, if you say you use the same job for matrix write + load + averaging, then you are better off allocating a single matrix and perform averaging in the C++ code directly allocate_and_zero_a single matrix for i in range(number_of_matrices) add_values_to_the_matrix scale_matrix_for_average Il giorno gio 22 mag 2025 alle ore 18:55 superdduc...@gmail.com < superdduc...@gmail.com> ha scritto: > Hi > > The averaging takes place on the same hardware and as part of the same job > as the writing the matrix to file in the c++ code. The number of nodes is > determined on the problem size and therefor for a single matrix. Allocating > extra nodes for the averaging is not economical, unfortunately. > > On 22 May 2025, at 16:52, Junchao Zhang <junchao.zh...@gmail.com> wrote: > > > Did you run in MPI parallel? If not, using MPI and running with multiple > compute nodes could solve the problem. > > Are all these matrices already on disk? Then you have to pay the I/O cost > for reading the matrices. > > --Junchao Zhang > > > On Thu, May 22, 2025 at 8:21 AM Donald Duck <superdduc...@gmail.com> > wrote: > >> Hello everyone >> >> A piece of c++ code writes PETSc AIJ matrices to binary files. My task is >> to compute an average matrix of these AIJ matrices. Therefore I read the >> first matrix with petsc4py and then start to add the other matrices to it. >> All matrixes always have the same size, shape, nnz etc. >> >> However in some cases these matrices are large and only one of it fits >> into memory and the reading/writing takes a significant amout of time. Is >> there a way to read it row by row to prevent memory overflow? >> >> I'm also open for other suggestions, thanks in advance! >> >> Raphael >> > -- Stefano
