Re: Applying synonyms increase the data size from MB to GBs

2011-06-06 Thread Erick Erickson
Have you considered query-time expansion rather than index-time expansion? In general this will lead to more complex queries, but smaller indexes. Take a look at the analysis page available from the admin page to see exactly what happens. What is the high-legel problem you're trying to solve? Hav

Re: Applying synonyms increase the data size from MB to GBs

2011-06-06 Thread Ahmet Arslan
> Is there a way where in I can apply all those file to same > tag with some > delimiter separated? > > like this: >         class="solr.SynonymFilterFactory" > synonyms="BODYTaxonomy.txt > , ClinicalObs.txt, MicTaxo.txt, SPTaxo.txt" > ignoreCase="true" > expand="true"/> Yes, you can perfectly

Re: Applying synonyms increase the data size from MB to GBs

2011-06-05 Thread pravesh
Since you r using expand="true" , so, every time a matching synonym entry is found the analyzer is expanding the term with all synonyms set in the index. This may cause the index to grow in size. -- View this message in context: http://lucene.472066.n3.nabble.com/Applying-synonyms-increase-the-da

Re: Applying synonyms increase the data size from MB to GBs

2011-06-05 Thread rajini maski
I have the flat files (synonym text files) each upto 200kb. Integrationg all of them increased the txt file size to huge. And I wanted to maintain them separately. So in order to apply all those synonyms to same field type I created that many filter tags for respective synonym txt files. Is it

Re: Applying synonyms increase the data size from MB to GBs

2011-06-05 Thread Gora Mohanty
On Mon, Jun 6, 2011 at 10:34 AM, rajini maski wrote: > Applying synonyms increased the data size from 28 mb to 10.3 gb > >   Before enabling synonyms to the a field , the data size was 28mb.  Now , > after applying synonyms I see that data folder size has increased to 10.3 > gb. > > Attached is sc