Re: Injecting synonymns into Solr

2015-05-04 Thread Zheng Lin Edwin Yeo
Yes, the underlying mechanism uses java. But the collection isn't able to load when the Solr starts up, so it didn't return anything even if I use url. Is it just due to my machine not having enough memory? Regards, Edwin On 4 May 2015 20:12, "Roman Chyla" wrote: > It shouldn't matter. Btw try

Re: Injecting synonymns into Solr

2015-05-04 Thread Shawn Heisey
On 5/4/2015 12:07 AM, Zheng Lin Edwin Yeo wrote: > Would like to check, will this method of splitting the synonyms into > multiple files use up a lot of memory? > > I'm trying it with about 10 files and that collection is not able to be > loaded due to insufficient memory. > > Although currently

Re: Injecting synonymns into Solr

2015-05-04 Thread Roman Chyla
It shouldn't matter. Btw try a url instead of a file path. I think the underlying loading mechanism uses java File , it could work. On May 4, 2015 2:07 AM, "Zheng Lin Edwin Yeo" wrote: > Would like to check, will this method of splitting the synonyms into > multiple files use up a lot of memory?

Re: Injecting synonymns into Solr

2015-05-03 Thread Zheng Lin Edwin Yeo
Would like to check, will this method of splitting the synonyms into multiple files use up a lot of memory? I'm trying it with about 10 files and that collection is not able to be loaded due to insufficient memory. Although currently my machine only have 4GB of memory, but I only have 500,000 rec

Re: Injecting synonymns into Solr

2015-04-30 Thread Zheng Lin Edwin Yeo
Thank you for the info. Yup this works. I found out that we can't load files that are more than 1MB into zookeeper, as it happens to any files that's larger than 1MB in size, not just the synonyms files. But I'm not sure if there will be an impact to the system, as the number of synonym text file c

Re: Injecting synonymns into Solr

2015-04-30 Thread Philippe Soares
Split your synonyms into multiple files and set the SynonymFilterFactory with a coma-separated list of files. e.g. : synonyms="syn1.txt,syn2.txt,syn3.txt" On Thu, Apr 30, 2015 at 8:07 PM, Zheng Lin Edwin Yeo wrote: > Just to populate it with the general synonym words. I've managed to > populate

Re: Injecting synonymns into Solr

2015-04-30 Thread Zheng Lin Edwin Yeo
Just to populate it with the general synonym words. I've managed to populate it with some source online, but is there a limit to what it can contains? I can't load the configuration into zookeeper if the synonyms.txt file contains more than 2100 lines. Regards, Edwin On 1 May 2015 05:44, "Chris H

Re: Injecting synonymns into Solr

2015-04-30 Thread Chris Hostetter
: There is a possible solution here: : https://issues.apache.org/jira/browse/LUCENE-2347 (Dump WordNet to SOLR : Synonym format). If you have WordNet synonyms you do't need any special code/tools to convert them -- the current solr.SynonymFilterFactory supports wordnet files (just specify forma

Re: Injecting synonymns into Solr

2015-04-30 Thread Zheng Lin Edwin Yeo
I'm using Solr-5.0.0 and ZooKeeper-3.4.6. I've gotton some samples from the Moby Treasure List http://www.gutenberg.org/catalog/world/results?title=moby+list to try it out. However, currently I can only have up to around 2100 lines in my synonyms.txt in when I load the configuration into ZooKeepe

Re: Injecting synonymns into Solr

2015-04-30 Thread Vincenzo D'Amore
Which version of solr? On Thu, Apr 30, 2015 at 9:58 AM, Zheng Lin Edwin Yeo wrote: > Hi, > > Does anyone knows any faster method of populating the synonyms.txt file > instead of manually typing in the words into the file, which there could be > thousands of synonyms around? > > Regards, > Edwin

Re: Injecting synonymns into Solr

2015-04-30 Thread Scott Dawson
There is a possible solution here: https://issues.apache.org/jira/browse/LUCENE-2347 (Dump WordNet to SOLR Synonym format). I don't have personal experience with it. I only know about it because it's mentioned on page 184 of the 'Solr in Action' book by Trey Grainger and Timothy Potter. Maybe som

Re: Injecting synonymns into Solr

2015-04-30 Thread Kaushik
I am facing the same problem; currently I am resorting to a custom program to create this file. Hopefully there is a better solution out there. Thanks, Kaushik On Thu, Apr 30, 2015 at 3:58 AM, Zheng Lin Edwin Yeo wrote: > Hi, > > Does anyone knows any faster method of populating the synonyms.tx

Injecting synonymns into Solr

2015-04-30 Thread Zheng Lin Edwin Yeo
Hi, Does anyone knows any faster method of populating the synonyms.txt file instead of manually typing in the words into the file, which there could be thousands of synonyms around? Regards, Edwin