The new replication feature does not use any unix commands , it is pure java. On the fly compression is hard but possible. I wish to repeat the question. Did you optimize the index? Because a 10:1 compression is not usually observed in an optimized index. Our own experiments showed compression of around 10:6 for optimized indexes.
--Noble On Wed, Oct 29, 2008 at 3:41 AM, Lance Norskog <[EMAIL PROTECTED]> wrote: > Aha! The hint to the actual problem: "When compressed with winzip". You are > running Solr on Windows. > > Snapshots don't work on Windows: they depend on a Unix file system feature. > You may be copying the entire index. Not just that, it could be inconsistent. > This is a fine topic for a "best practices for Windows" wiki page. > > The 'scp' program what you want. It has an option to compress on the fly > without saving anything to disk. 'Rcopy' in particular has features to only > copy what is not already at the target. The Putty suite 'pscp' program also > has the compression feature. > > Lance > > -----Original Message----- > From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] > Sent: Monday, October 27, 2008 9:36 PM > To: solr-user@lucene.apache.org > Subject: Re: replication handler - compression > >> It is useful only if your bandwidth is very low. >> Otherwise the cost of copying/comprressing/decompressing can take up >> more time than we save. > > I mean compressing and transferring. If the optimized index itself has a very > high compression ratio then it is worth exploring the option of compresssing > and transferring. And do not assume that all the files in the index directory > is transferred during replication. It only transfers the files which are used > by the current commit point and the ones which are absent in the slave > > >> >> >> >> On Tue, Oct 28, 2008 at 2:49 AM, Simon Collins >> <[EMAIL PROTECTED]> wrote: >>> Is there an option on the replication handler to compress the files? >>> >>> >>> >>> I'm trying to replicate off site, and seem to have accumulated about >>> 1.4gb. When compressed with winzip of all things i can get this down >>> to about 10% of the size. >>> >>> >>> >>> Is compression in the pipeline / can it be if not! >>> >>> >>> >>> simon >>> >>> >>> >>> This message has been scanned for malware by SurfControl plc. >>> www.surfcontrol.com >>> >> >> >> >> -- >> --Noble Paul >> > > > > -- > --Noble Paul > > -- --Noble Paul
