Yes sure, thanks for your advice. I'm still waiting for my server to come before I can scale up my system and do the testing. Now the Solr running on my 4GB RAM system will crash if I try to scale up my system as there's not enough memory to support it.
Regards, Edwin On 11 May 2015 at 19:11, Alessandro Benedetti <benedetti.ale...@gmail.com> wrote: > 2015-05-11 4:44 GMT+01:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com>: > > > I've managed to run the synonyms with 10 different synonyms file. Each of > > the synonym file size is 1MB, which consist of about 1000 tokens, and > each > > token has about 40-50 words. These lists of files are more extreme, > which I > > probably won't use for the real environment, except now for the testing > > purpose. > > > > The QTime is about 100-200, as compared to about 50 for collection > without > > synonyms configured. > > > > Is this timing consider fast or slow? Although the synonyms files are > big, > > there's not that many index in my collection yet. Just afraid the > > performance will be affected when more index comes in. > > > > If it's fast or slow it depends on your requirements :) > For a human waiting for the response, I would say 100ms to be quite fast. > To understand what happens when the index scale up, you should prototype ! > Anyway there are a lot of solution in Solr to scale up your system ! > > Cheers > > > > > Regards, > > Edwin > > On 9 May 2015 00:14, "Zheng Lin Edwin Yeo" <edwinye...@gmail.com> > wrote: > > > > > Thank you for your suggestions. > > > > > > I can't do a proper testing on that yet as I'm currently using a 4GB > RAM > > > normal PC machine, and all these probably requires more RAM that what I > > > have. > > > I've tried running the setup with 20 synonyms file, and the system went > > > Out of Memory before I could test anything. > > > > > > For your option 2), do you mean that I'll need to download a synonym > > > database (like the one with over 20MB in size which I have), and index > > them > > > into an Ad Hoc Solr Core to manage them? > > > > > > I probably can only try them out properly when I can get the server > > > machine with more RAM. > > > > > > Regards, > > > Edwin > > > > > > > > > On 8 May 2015 at 22:16, Alessandro Benedetti < > benedetti.ale...@gmail.com > > > > > > wrote: > > > > > >> This is a quite big Sinonym corpus ! > > >> If it's not feasible to have only 1 big synonym file ( I haven't > > checked, > > >> so I assume the 1 Mb limit is true, even if strange) > > >> I would do an experiment : > > >> 1) testing query time with a Solr Classic config > > >> 2) Use an Ad Hoc Solr Core to manage Synonyms ( in this way we can > keep > > it > > >> updated and use it with a custom version of the Sysnonym filter that > > will > > >> get the Synonyms directly from another Solr instance). > > >> 2b) develop a Solr plugin to provide this approach > > >> > > >> If the synonym thesaurus is really big, I guess managing them through > > >> another Solr Core ( or something similar) locally , will be better > than > > >> managing it with an external web service. > > >> > > >> Cheers > > >> > > >> 2015-05-08 12:16 GMT+01:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com > >: > > >> > > >> > So it means like having more than 10 or 20 synonym files locally > will > > >> still > > >> > be faster than accessing external service? > > >> > > > >> > As I found out that zookeeper only allows the synonym.txt file to > be a > > >> > maximum of 1MB, and as my potential synonym file is more than 20MB, > > I'll > > >> > need to split the file to more than 20 of them. > > >> > > > >> > Regards, > > >> > Edwin > > >> > > > >> > > >> > > >> > > >> -- > > >> -------------------------- > > >> > > >> Benedetti Alessandro > > >> Visiting card : http://about.me/alessandro_benedetti > > >> > > >> "Tyger, tyger burning bright > > >> In the forests of the night, > > >> What immortal hand or eye > > >> Could frame thy fearful symmetry?" > > >> > > >> William Blake - Songs of Experience -1794 England > > >> > > > > > > > > > > > > -- > -------------------------- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
