I got it, thanks so much! Eda
On Wed, Jul 1, 2020, 2:42 PM Matthew Knepley <knep...@gmail.com> wrote: > On Wed, Jul 1, 2020 at 7:32 AM Eda Oktay <eda.ok...@metu.edu.tr> wrote: > >> I'm sorry but I still can't understand how to put an input as these >> parameters. It is the first time I saw 'const' for IS. If it doesn't >> refer IS, then as a parameter for row and column indices, how should I >> give them as a valid input? It still should be an IS, right? >> >> Lastly, I tried to use in the form: >> >> IS idUi; >> ISCreateStride(PETSC_COMM_WORLD,siz,0,1,&idUi); >> Mat *submat; >> MatCreateSubMatrices(U,nev,&idUi,&idUi,MAT_INITIAL_MATRIX,&submat); >> > > You gave n = 4 (you call it nev), but you only have 1 rowIS. You are > supposed to give an array of 4. > > Thanks, > > Matt > > >> where nev = 4, siz = 72 and U is 72*4 matrix. >> >> But then, the error becomes about the matrix, U: >> >> [0]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message >> -------------------------------------------------------------- >> [1]PETSC ERROR: Invalid argument >> [1]PETSC ERROR: Wrong type of object: Parameter # 1 >> Invalid argument >> [0]PETSC ERROR: Wrong type of object: Parameter # 1 >> [0]PETSC ERROR: See >> https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >> shooting. >> [0]PETSC ERROR: Petsc Release Version 3.13.2, Jun 02, 2020 >> [0]PETSC ERROR: ./deneme_new_vecscatter_arastep on a >> arch-linux2-c-debug named b342.wls.metu.edu.tr by edaoktay Wed Jul 1 >> 14:25:00 2020 >> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >> --download-mpich --download-openblas --download-slepc --download-metis >> --download-parmetis --download-chaco --with-X=1 >> [1]PETSC ERROR: See >> https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble >> shooting. >> [1]PETSC ERROR: Petsc Release Version 3.13.2, Jun 02, 2020 >> [1]PETSC ERROR: [0]PETSC ERROR: #1 ISSorted() line 1777 in >> /home/edaoktay/petsc-3.13.2/src/vec/is/is/interface/index.c >> [0]PETSC ERROR: ./deneme_new_vecscatter_arastep on a >> arch-linux2-c-debug named b342.wls.metu.edu.tr by edaoktay Wed Jul 1 >> 14:25:00 2020 >> [1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ >> --download-mpich --download-openblas --download-slepc --download-metis >> --download-parmetis --download-chaco --with-X=1 >> [1]PETSC ERROR: #2 MatCreateSubMatrices_MPIDense_Local() line 104 in >> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c >> [0]PETSC ERROR: #3 MatCreateSubMatrices_MPIDense() line 66 in >> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c >> #1 ISSorted() line 1777 in >> /home/edaoktay/petsc-3.13.2/src/vec/is/is/interface/index.c >> [1]PETSC ERROR: #2 MatCreateSubMatrices_MPIDense_Local() line 104 in >> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c >> [0]PETSC ERROR: #4 MatCreateSubMatrices() line 6758 in >> /home/edaoktay/petsc-3.13.2/src/mat/interface/matrix.c >> [1]PETSC ERROR: #3 MatCreateSubMatrices_MPIDense() line 66 in >> /home/edaoktay/petsc-3.13.2/src/mat/impls/dense/mpi/mmdense.c >> [1]PETSC ERROR: #4 MatCreateSubMatrices() line 6758 in >> /home/edaoktay/petsc-3.13.2/src/mat/interface/matrix.c >> >> Before I used &idUi, I tried to use just idUi (without & sign) and it >> didn't work, so I decided to try using this way, since may be 'const' >> refers this one. But how can my matrix be of wrong type? It is MPI >> Dense. >> >> Thanks! >> >> Eda >> >> Matthew Knepley <knep...@gmail.com>, 1 Tem 2020 Çar, 13:43 tarihinde >> şunu yazdı: >> > >> > On Wed, Jul 1, 2020 at 6:34 AM Eda Oktay <eda.ok...@metu.edu.tr> wrote: >> >> >> >> Der Barry, >> >> >> >> ı am trying to use your way but I couldn't understand how I create sub >> >> matrices by using MatCreateSubMatrices() since as input, the function >> >> needs const IS instead of IS and I couldn't understand how to get a >> >> const IS. I tried to use ISCreateStride since this IS should be 0:71 >> >> because as you mentioned, the sub matrix should consist of the entire >> >> matrix. However, since ISCreateStride produces IS, not const IS, I >> >> couldn't use it in MatCreateSubMatrices(). >> > >> > >> > The 'const' refers to the array, not the IS: >> > >> > >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateSubMatrices.html >> > >> > Thanks, >> > >> > Matt >> > >> >> >> >> Thanks >> >> >> >> Eda >> >> >> >> Barry Smith <bsm...@petsc.dev>, 10 Haz 2020 Çar, 19:59 tarihinde şunu >> yazdı: >> >> > >> >> > >> >> > You can use MatCreateSubMatrices() with each process getting a >> single sequential dense sub matrix that consists of the entire matrix. >> >> > >> >> > Use VecDuplicateVecs() to create an array of 72 vectors (create a >> single seq vector of size 4 as the input to this routine) >> >> > >> >> > Then use MatDenseGetArrayRead() to access the upper left corner of >> the new sequential dense matrix >> >> > >> >> > Loop over the vectors calling VecGetArray() >> >> > Then loop over the row of the dense array filling up the vector >> >> > >> >> > Because dense matrices are stored by column, you have to do this >> looping to fill up the vectors, they can't share the space with the matrix. >> >> > >> >> > Barry >> >> > >> >> > >> >> > >> >> > On Jun 10, 2020, at 11:36 AM, Matthew Knepley <knep...@gmail.com> >> wrote: >> >> > >> >> > On Wed, Jun 10, 2020 at 12:26 PM Eda Oktay <eda.ok...@metu.edu.tr> >> wrote: >> >> >> >> >> >> Matthew Knepley <knep...@gmail.com>, 10 Haz 2020 Çar, 19:13 >> tarihinde >> >> >> şunu yazdı: >> >> >> > >> >> >> > On Wed, Jun 10, 2020 at 12:07 PM Eda Oktay <eda.ok...@metu.edu.tr> >> wrote: >> >> >> >> >> >> >> >> Der Matt, >> >> >> >> >> >> >> >> When I looked at the results, I found that there are some >> problems I >> >> >> >> couldn't understand. >> >> >> >> >> >> >> >> First of all, I am working on a 72*4 matrix and as I said >> before, I >> >> >> >> want to have 72 different vectors having size 4 each, whose >> elements >> >> >> >> consist of the elements in the same row. And of course, all >> vectors >> >> >> >> should be in all processors (currently I am using 4 processors). >> >> >> >> >> >> >> >> When I use your scatter code, the output vector is divided into 4 >> >> >> >> parts for 4 processors and each vector consists of 18 row vectors >> >> >> >> whose elements are arranged in a way that if I want to find >> zeroth row >> >> >> >> vector, its elements are located in 0th,18th,36th,54th elements. >> >> >> > >> >> >> > >> >> >> > Was the global size of the vector you wrapped around the dense >> matrix 72*4? >> >> >> >> >> >> Yes it is. I set up its global size to 72*4. >> >> >> >> >> >> > >> >> >> > If you use CreateToAll(), it will make a vector on each process >> which has the global size of the original vector. >> >> >> >> >> >> Although I set 72*4, the size of the vectors in each process is 72. >> >> > >> >> > >> >> > You can understand how it is hard to accept, as this code is tested >> every night. Can you VecView() the input vector >> >> > to CreateToAll and the output vector, and send that output? >> >> > >> >> > Thanks, >> >> > >> >> > Matt >> >> > >> >> >> >> >> >> Thanks, >> >> >> >> >> >> Eda >> >> >> >> >> >> > >> >> >> > Thanks, >> >> >> > >> >> >> > Matt >> >> >> > >> >> >> >> >> >> >> >> So, isn't scatter's goal is to scatter all values to all >> processors? >> >> >> >> >> >> >> >> Furthermore, I am trying to use my vectors in that way but isn't >> there >> >> >> >> any possible way that I can reach my goal entirely? >> >> >> >> >> >> >> >> Thanks so much for your help, >> >> >> >> >> >> >> >> Eda >> >> >> >> >> >> >> >> Matthew Knepley <knep...@gmail.com>, 10 Haz 2020 Çar, 18:11 >> tarihinde >> >> >> >> şunu yazdı: >> >> >> >> > >> >> >> >> > On Wed, Jun 10, 2020 at 10:09 AM Eda Oktay < >> eda.ok...@metu.edu.tr> wrote: >> >> >> >> >> >> >> >> >> >> Dear Matt, >> >> >> >> >> >> >> >> >> >> I have one last question I believe. Up to creating a dense >> matrix I >> >> >> >> >> did what you've suggested. Thank you so much for that. >> >> >> >> >> >> >> >> >> >> I created a new dense matrix. Now, how should I wrap each >> vector in a >> >> >> >> >> MatDense again? I mean, what is wrapping vectors in a matrix? >> To put >> >> >> >> >> each of them again as rows? >> >> >> >> > >> >> >> >> > >> >> >> >> > I thought you need a dense matrix for something, since you >> started with one. If you >> >> >> >> > do not, just do VecGetArray() on the vector from CreateToAll >> and use the values. >> >> >> >> > >> >> >> >> > Thanks, >> >> >> >> > >> >> >> >> > Matt >> >> >> >> > >> >> >> >> >> >> >> >> >> >> Thanks! >> >> >> >> >> >> >> >> >> >> Eda >> >> >> >> >> >> >> >> >> >> Matthew Knepley <knep...@gmail.com>, 10 Haz 2020 Çar, 16:16 >> tarihinde >> >> >> >> >> şunu yazdı: >> >> >> >> >> > >> >> >> >> >> > On Wed, Jun 10, 2020 at 9:08 AM Eda Oktay < >> eda.ok...@metu.edu.tr> wrote: >> >> >> >> >> >> >> >> >> >> >> >> Dear Matt, >> >> >> >> >> >> >> >> >> >> >> >> Matthew Knepley <knep...@gmail.com>, 10 Haz 2020 Çar, >> 16:03 tarihinde >> >> >> >> >> >> şunu yazdı: >> >> >> >> >> >> > >> >> >> >> >> >> > On Wed, Jun 10, 2020 at 8:56 AM Eda Oktay < >> eda.ok...@metu.edu.tr> wrote: >> >> >> >> >> >> >> >> >> >> >> >> >> >> Hi all, >> >> >> >> >> >> >> >> >> >> >> >> >> >> I am trying to get all the rows of a parallel matrix as >> individual >> >> >> >> >> >> >> vectors. For instance, if I have 72*4 matrix, I want to >> get 72 >> >> >> >> >> >> >> different vectors having size 4. >> >> >> >> >> >> >> >> >> >> >> >> >> >> As far as I understood, MatGetRow is only for local >> rows, so >> >> >> >> >> >> >> MatGetOwnershipRange is used, however, when I tried >> this one, I >> >> >> >> >> >> >> couldn't get the whole and desired row vectors. >> >> >> >> >> >> >> >> >> >> >> >> >> >> In MatGetRow explanation, it is written that I should >> use >> >> >> >> >> >> >> MatCreateSubMatrices first, then use MatGetRow. But I >> couldn't >> >> >> >> >> >> >> understand to which extent I should create submatrices. >> I just need to >> >> >> >> >> >> >> have all 72 rows as 72 different vectors each having 4 >> elements. >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > 1) For sparse matrices, the storage is always divided by >> row, so that values can only be retrieved for local rows with MatGetRow() >> >> >> >> >> >> > >> >> >> >> >> >> > 2) Is this matrix sparse? It sounds like it is dense. >> >> >> >> >> >> >> >> >> >> >> >> Matrix is dense. >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> >> >> > 3) Are you asking to get all matrix values on all >> processes? If so, I think the easiest thing to do is first wrap a Vec >> around the >> >> >> >> >> >> > values, then use VecScatterToAll(), then wrap each >> one in a MatDense again. >> >> >> >> >> >> >> >> >> >> >> >> Yes, I want all row vectors on all processes. In a dense >> matrix, >> >> >> >> >> >> should I still wrap a Vec around the values? I know I >> should use >> >> >> >> >> >> scatter but I couldn't even wrap a Vec around them. >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > I would do >> >> >> >> >> > >> >> >> >> >> > MatGetSize(&N); >> >> >> >> >> > MatGetLocalSize(&m); >> >> >> >> >> > >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatDenseGetArray.html >> >> >> >> >> > <create vector of local size m*N> >> >> >> >> >> > >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecPlaceArray.html >> >> >> >> >> > >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecScatterCreateToAll.html >> >> >> >> >> > <do scatter> >> >> >> >> >> > >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecResetArray.html#VecResetArray >> >> >> >> >> > >> https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateDense.html >> >> >> >> >> > <use it> >> >> >> >> >> > <destroy matrix> >> >> >> >> >> > <destroy vector from CreateToAll> >> >> >> >> >> > >> >> >> >> >> > Thanks, >> >> >> >> >> > >> >> >> >> >> > Matt >> >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> Thanks so much! >> >> >> >> >> >> >> >> >> >> >> >> Eda >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> >> >> > Thanks, >> >> >> >> >> >> > >> >> >> >> >> >> > Matt >> >> >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> >> >> Thanks! >> >> >> >> >> >> >> >> >> >> >> >> >> >> Eda >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > >> >> >> >> >> >> > -- >> >> >> >> >> >> > What most experimenters take for granted before they >> begin their experiments is infinitely more interesting than any results to >> which their experiments lead. >> >> >> >> >> >> > -- Norbert Wiener >> >> >> >> >> >> > >> >> >> >> >> >> > https://www.cse.buffalo.edu/~knepley/ >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > -- >> >> >> >> >> > What most experimenters take for granted before they begin >> their experiments is infinitely more interesting than any results to which >> their experiments lead. >> >> >> >> >> > -- Norbert Wiener >> >> >> >> >> > >> >> >> >> >> > https://www.cse.buffalo.edu/~knepley/ >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > -- >> >> >> >> > What most experimenters take for granted before they begin >> their experiments is infinitely more interesting than any results to which >> their experiments lead. >> >> >> >> > -- Norbert Wiener >> >> >> >> > >> >> >> >> > https://www.cse.buffalo.edu/~knepley/ >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> >> >> > -- Norbert Wiener >> >> >> > >> >> >> > https://www.cse.buffalo.edu/~knepley/ >> >> > >> >> > >> >> > >> >> > -- >> >> > What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> >> > -- Norbert Wiener >> >> > >> >> > https://www.cse.buffalo.edu/~knepley/ >> >> > >> >> > >> > >> > >> > >> > -- >> > What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> > -- Norbert Wiener >> > >> > https://www.cse.buffalo.edu/~knepley/ >> > > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <http://www.cse.buffalo.edu/~knepley/> >
