SVN 1.6.17 dump is growing larger than repository size (approx. more than 10 times)
Hello Team, I am creating Subversion 1.6.17 dump for a repository hosted in Linux server. SVNSERVE is serving the repository. We are migrating to SVN 1.9.2. *While creating the dump for repository of size 1.8 GB (revisions 3000+), the dump command completes revision 119 and hangs and keep updating the dumpfile which grows to 7-8 GBs. dump command is not moving to next revision but kept updating the dump file. So, i just closed the putty to stop creating the dump.* However, I ran the svnadmin verify which completes revision 119 and take some time to verify revision 120 and complete the verification successfully for all the repository revisions. Since, there is no error output I have no logs to attach. Please suggest the work around for this situation. How can create the dump and migrate to target server. I am planning to use rsync from the server. I will post how it goes. I am expecting the workaround to this situation and let me know if this is a known issue in SVN 1.6.17. This is my active repository and created the dump without stopping the svnserve program in the production server. Thanks, Arun
Re: SVN 1.6.17 dump is growing larger than repository size (approx. more than 10 times)
Hi Andreas, I have used svnsync and it took a while for rev 120 but it went successfully. There is no space constraint but it was taking the longer time when compared to other repositories. So for now, i have migrated most repositories and will see how i am going to migrate the rest in coming months. rsync is not useful for me since I am moving from one to another server without altering the existing setup. For -M options - is this is the cache for processing for dump files. do I need to increase the value in MB like -M 1024 to speed up. Sorry for late reply. regards, Arun On Wed, Nov 25, 2015 at 4:31 PM, Andreas Stieger wrote: > Hello, > > Yes the dump can be significantly larger than the repository size, due to > delta encoding used in the on-disk representation. If you examine revision > 120 I am sure you will fine a particularly complex change that would cause > this, and your statement about 120 being larger than normal is in line with > that. > > This is not a problem, it is simply expected behaviour. If the size of the > dumpstream is a problem for you, you can use "--deltas" to reduce size of > the dump stream. > > If things are slow for you, specify a memory cache size (e.g. 1-2 GB if > you have it) to speed up operations. > > -M [--memory-cache-size] ARG : size of the extra in-memory cache in MB > used to > minimize redundant operations. Default: 16. > [used for FSFS repositories only] > > You can use svnsync for a more transparent migration. I am not sure how > rsync should be relevant or useful here when doing dumps anyway. > > Andreas > > *Gesendet:* Mittwoch, 25. November 2015 um 10:25 Uhr > *Von:* "arun prasath" > *An:* users@subversion.apache.org > *Betreff:* SVN 1.6.17 dump is growing larger than repository size > (approx. more than 10 times) > Hello Team, > > I am creating Subversion 1.6.17 dump for a repository hosted in Linux > server. SVNSERVE is serving the repository. We are migrating to SVN 1.9.2. > > *While creating the dump for repository of size 1.8 GB (revisions 3000+), > the dump command completes revision 119 and hangs and keep updating the > dumpfile which grows to 7-8 GBs. dump command is not moving to next > revision but kept updating the dump file. So, i just closed the putty to > stop creating the dump.* > > However, I ran the svnadmin verify which completes revision 119 and take > some time to verify revision 120 and complete the verification successfully > for all the repository revisions. > > Since, there is no error output I have no logs to attach. Please suggest > the work around for this situation. How can create the dump and migrate to > target server. I am planning to use rsync from the server. I will post how > it goes. > > I am expecting the workaround to this situation and let me know if this is > a known issue in SVN 1.6.17. This is my active repository and created the > dump without stopping the svnserve program in the production server. > > > Thanks, > Arun >
Re: SVN 1.6.17 dump is growing larger than repository size (approx. more than 10 times)
I will use this and give a try. Thank you very much for help information. regards, Arun On Sat, Dec 12, 2015 at 9:12 PM, Andreas Stieger wrote: > On 12/12/15 13:54, arun prasath wrote: > >> For -M options - is this is the cache for processing for dump files. do I >> need to increase the value in MB like -M 1024 to speed up. >> > > Yes this affects processing dump files. This option is documented. It > increases the size of the memory cache used to reduce redundant operations. > Give it plenty if you have it. > > Andreas >
Regarding install and running svn1.9.2 for svnserve and http access in browser
Hi Team, I have situation which needs your suggestion and help in setting up the svn1.9.2 with svnserve + http access in browser. My below talks about my svn setup. *SVN working method:* After installation, the svn://servername/reponame is working. But http://servername/reponame is not working and give the HTTP 404 error. *Problem Description:* Am i missing something. I know Svn Apache modules are required for http access but i am running svnserve for client access and no Apache httpd is installed. For http access i have installed Serf library. *Expected : * - What are required for running svn1.9.2 as svnserve daemon (with http support in browser) -Is just serf library is not enough to serve http through svnserve -how to pass the httpd modules to during subversion install Please suggest how to proceed in this situation. I have already installed the svn without serf in production and now would like to configure svn with http support in the custom path. Thank you. *ENV Description* *Redhat Linux* - Redhat 7 *SVN install type* - Install from Source/TAR ball *Install to Custom path* - Like /local/application/packages/svn1.9.2 *TAR ball path* - /local/application/source/subversion-1.9.2 *SVN access method(as daemon)* - /local/application/packages/svn1.9.2/bin/svnserve -d -r *Pre-requisite installed location* - to custom path like svn *Pre-requisite installed* are - apr-1.5.2, apr-util-1.5.4, serf-1.3.8,scons-2.4.1,Python-2.7.11,openssl-1.0.2e [ all source/tar install from source to custom path] *Path set* - /local/application/pacakges/source/scons/script to PATH variable. I have set the scons custom install path but svn configure script is not taking the libserf-1.so. so set the serf source path to script folder in PATH variable and it takes the serf. *svn1.9.2 install options* ./configure --prefix=/local/application/packages/svn-1.9.2 --with-serf=/local/application/packages/serf1.3.8 --with-apr=/local/application/packages/apr-config-1 --with-apr-util=/local/application/packages/apr-config-1 --disable-static --with-sqlite=/local/application/pacakges/source/subversion-1.9.2/sqlite-amalgamation-3081101/sqlite3.c make make install regards, Arun
Re: Regarding install and running svn1.9.2 for svnserve and http access in browser
Hi Ryan, Thanks for the info. I am running svnserve for client access through TortoiseSVN. This is for users who wants to checkout and checkin the code. There are some users who just want to browse the code in Internet Explorer. The http in browser is required for users who browse code without checkout using TortoiseSVN client. How to achieve this. So, I have installed svn using configure.sh script i have used the serf library path. But serf is not serving the http for svn repository. So my doubt is whether the following option will provide http support when svn is installed with below options. I hope this explains what my requirement is. --with-apxs[=FILE] Build shared Apache modules. FILE is the optional pathname to the Apache apxs tool; defaults to "apxs". --with-apache-libexecdir[=PATH] Install Apache modules to Apache's configured modules directory instead of LIBEXECDIR; if PATH is given, install to PATH. regards, Arun On Fri, Dec 18, 2015 at 12:04 AM, Ryan Schmidt < subversion-2...@ryandesign.com> wrote: > > On Dec 17, 2015, at 8:09 AM, arun prasath wrote: > > > I have situation which needs your suggestion and help in setting up the > svn1.9.2 with svnserve + http access in browser. My below talks about my > svn setup. > > SVN working method: > > After installation, the svn://servername/reponame is working. But > http://servername/reponame is not working and give the HTTP 404 error. > > > > Problem Description: > > Am i missing something. I know Svn Apache modules are required for http > access but i am running svnserve for client access and no Apache httpd is > installed. For http access i have installed Serf library. > > > > Expected : > > - What are required for running svn1.9.2 as svnserve daemon (with http > support in browser) > > -Is just serf library is not enough to serve http through svnserve > > -how to pass the httpd modules to during subversion install > > Apache + mod_dav_svn is required to serve a Subversion repository over > http (or https). You can run this simultaneously with svnserve for serving > the repository over the svn protocol, provided you set up > permissions/ownership properly. There's documentation in the book. However, > I'm not sure what benefit there is to offering multiple access methods. For > example, if you want to serve the repository over http via Apache + > mod_dav_svn, then why also run svnserve? > > serf is the library the svn client uses to access a repository that is > being served by Apache + mod_dav_svn. > > >
Re: Regarding install and running svn1.9.2 for svnserve and http access in browser
Hi Ryan, Sure will add the Team... *One last question.* I have already setup the SVN using compile options using apr, apr-util and sqlite custom install and my svnserve is running. So, for using http for READ only access to svn repository path, Installing Apache with mod_dav_svn should provide access to svn repository. (OR ) Do I need to install Subversion with Apache APR modules. I hope this is not required since mod_dav_svn will talk to repository and provide read access through http. What I understood is, Configuring Apache http(2.2.14 and above preferably) and having mod_dav_svn, mod_*_authz packages should server my repository path for READ only access to SVN repository. For this I don't need Subversion installed. Please correct me If my understanding is correct. I am asking this question since Apache will be installed first and Subversion will be installed second with Apr installed in Apache. Thank you. regards, Arun On Wed, Dec 23, 2015 at 2:08 AM, Ryan Schmidt < subversion-2...@ryandesign.com> wrote: > Remember to Reply All so the conversation stays on the mailing list. I'm > re-adding the mailing list address to this reply. > > > On Dec 22, 2015, at 10:07 AM, arun prasath wrote: > > > Yes Ryan you are right. > > > > I am working on setting up Apache with mod_dav_svn for http access. > Users can not view the files in clients like TortoiseSVN unless it is > checkedout. So users want http access via browsers. > > FYI TortoiseSVN has a repository browser you can use to view files without > checking them out. > > > https://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-dug-repobrowser.html > > > > Thanks for your very valuable information. I thought of setting up http > support through Serf but now decided to setup Apache +mod_dav_svn. > > > > I have sometime to google and find right command and options to install > Apache for Subversion. If all goes well, will not comeback to svn Dev group. > > > > Thank you very much for help and addressing my concerns with patience. > > > > Regards, > > Arun > >
Re: Regarding install and running svn1.9.2 for svnserve and http access in browser
n_subr-1.so.0 (0x7fdee3862000) libz.so.1 => /lib64/libz.so.1 (0x7fdee364c000) libaprutil-1.so.0 => /local1/apps/pkg/apr-config/lib/libaprutil-1.so.0 (0x7fdee3426000) libexpat.so.1 => /lib64/libexpat.so.1 (0x7fdee31fb000) libapr-1.so.0 => /local1/apps/pkg/apr-config/lib/libapr-1.so.0 (0x7fdee2fc8000) libuuid.so.1 => /lib64/libuuid.so.1 (0x7fdee2dc3000) librt.so.1 => /lib64/librt.so.1 (0x7fdee2bba000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x7fdee2983000) libpthread.so.0 => /lib64/libpthread.so.0 (0x7fdee2767000) libdl.so.2 => /lib64/libdl.so.2 (0x7fdee2562000) libc.so.6 => /lib64/libc.so.6 (0x7fdee21a1000) libresolv.so.2 => /lib64/libresolv.so.2 (0x7fdee1f86000) /lib64/ld-linux-x86-64.so.2 (0x7fdee58cf000) libfreebl3.so => /lib64/libfreebl3.so (0x7fdee1d82000) *Apache version and configuration where I have compiled svn modules in test env * *Apr* ./configure --prefix=/local/apps/pkgs/apr-config-2 --disable-static --with-installbuilddir=/local/apps/pkgs/apr-config-2/build *Apr-util* ./configure --prefix=/local/apps/pkgs/apr-config-2 --with-apr=/local/apps/pkgs/apr-config-2 --with-gdbm=/local/apps/pkgs/apr-config-2 --with-openssl=/local/apps/pkgs/openssl-1.0.2e --with-crypto --with-ldap *Apache* ./configure --prefix=/local/apps/pkgs/apache/httpd2.4.18_ldap --enable-authnz-ldap -enable-mpms-shared=all --with-apr=/local/apps/pkgs/apr-config-2 --with-apr-util=/local/apps/pkgs/apr-config-2 --with-pcre=/local/apps/pkgs/pcre-8.38 Apache configuration: -bash-4.1$ /local/apps/pkgs/apache/httpd2.4.18_ldap/bin/httpd -V Server version: Apache/2.4.18 (Unix) Server built: Jan 7 2016 05:03:44 Server's Module Magic Number: 20120211:52 Server loaded: APR 1.5.2, APR-UTIL 1.5.4 Compiled using: APR 1.5.2, APR-UTIL 1.5.4 *Architecture: 64-bit* *Server MPM: event* threaded: yes (fixed thread count) forked: yes (variable process count) Server compiled with -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=256 -D HTTPD_ROOT="/local/apps/pkgs/apache/httpd2.4.18_ldap" -D SUEXEC_BIN="/local/apps/pkgs/apache/httpd2.4.18_ldap/bin/suexec" -D DEFAULT_PIDLOG="logs/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" Regards, Arun On Thu, Dec 24, 2015 at 9:59 AM, Ryan Schmidt < subversion-2...@ryandesign.com> wrote: > > On Dec 23, 2015, at 5:26 AM, arun prasath wrote: > > > I have already setup the SVN using compile options using apr, apr-util > and sqlite custom install and my svnserve is running. > > > > So, for using http for READ only access to svn repository path, > Installing Apache with mod_dav_svn should provide access to svn repository. > > (OR ) Do I need to install Subversion with Apache APR modules. I hope > this is not required since mod_dav_svn will talk to repository and provide > read access through http. > > > > What I understood is, Configuring Apache http(2.2.14 and above > preferably) and having mod_dav_svn, mod_*_authz packages should server my > repository path for READ only access to SVN repository. For this I don't > need Subversion installed. > > > > Please correct me If my understanding is correct. I am asking this > question since Apache will be installed first and Subversion will be > installed second with Apr installed in Apache. > > > mod_dav_svn is a part of the software called Subversion (or more properly: > Apache Subversion, since it is owned by the Apache Software Foundation). So > are the Subversion command line programs (including svn, svnadmin, svnlook, > svnsync and others) and the Subversion libraries (including libsvn_base, > libsvn_client, libsvn_delta and others). > > You don't need the Subversion command line programs to use Apache httpd > and mod_dav_svn, but you do need the Subversion libraries. You also need > mod_dav_svn to have been built for the same version of Apache httpd you're > going to run. This is easy if you build from source, but if you're > installing pre-compiled binaries make sure they're properly matched. > > Both Apache httpd and Apache Subversion require the Apache Portable > Runtime (apr) and the Apache Portable Runtime Utility (apr-util). It's > probably a good idea to use the same apr and apr-util libraries for > everything. Again, this will probably take care of itself if you build from > source, but verify this if you install a binary. > > >
