2015-05-11 4:44 GMT+01:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com>:
> I've managed to run the synonyms with 10 different synonyms file. Each of > the synonym file size is 1MB, which consist of about 1000 tokens, and each > token has about 40-50 words. These lists of files are more extreme, which I > probably won't use for the real environment, except now for the testing > purpose. > > The QTime is about 100-200, as compared to about 50 for collection without > synonyms configured. > > Is this timing consider fast or slow? Although the synonyms files are big, > there's not that many index in my collection yet. Just afraid the > performance will be affected when more index comes in. > If it's fast or slow it depends on your requirements :) For a human waiting for the response, I would say 100ms to be quite fast. To understand what happens when the index scale up, you should prototype ! Anyway there are a lot of solution in Solr to scale up your system ! Cheers > > Regards, > Edwin > On 9 May 2015 00:14, "Zheng Lin Edwin Yeo" <edwinye...@gmail.com> wrote: > > > Thank you for your suggestions. > > > > I can't do a proper testing on that yet as I'm currently using a 4GB RAM > > normal PC machine, and all these probably requires more RAM that what I > > have. > > I've tried running the setup with 20 synonyms file, and the system went > > Out of Memory before I could test anything. > > > > For your option 2), do you mean that I'll need to download a synonym > > database (like the one with over 20MB in size which I have), and index > them > > into an Ad Hoc Solr Core to manage them? > > > > I probably can only try them out properly when I can get the server > > machine with more RAM. > > > > Regards, > > Edwin > > > > > > On 8 May 2015 at 22:16, Alessandro Benedetti <benedetti.ale...@gmail.com > > > > wrote: > > > >> This is a quite big Sinonym corpus ! > >> If it's not feasible to have only 1 big synonym file ( I haven't > checked, > >> so I assume the 1 Mb limit is true, even if strange) > >> I would do an experiment : > >> 1) testing query time with a Solr Classic config > >> 2) Use an Ad Hoc Solr Core to manage Synonyms ( in this way we can keep > it > >> updated and use it with a custom version of the Sysnonym filter that > will > >> get the Synonyms directly from another Solr instance). > >> 2b) develop a Solr plugin to provide this approach > >> > >> If the synonym thesaurus is really big, I guess managing them through > >> another Solr Core ( or something similar) locally , will be better than > >> managing it with an external web service. > >> > >> Cheers > >> > >> 2015-05-08 12:16 GMT+01:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com>: > >> > >> > So it means like having more than 10 or 20 synonym files locally will > >> still > >> > be faster than accessing external service? > >> > > >> > As I found out that zookeeper only allows the synonym.txt file to be a > >> > maximum of 1MB, and as my potential synonym file is more than 20MB, > I'll > >> > need to split the file to more than 20 of them. > >> > > >> > Regards, > >> > Edwin > >> > > >> > >> > >> > >> -- > >> -------------------------- > >> > >> Benedetti Alessandro > >> Visiting card : http://about.me/alessandro_benedetti > >> > >> "Tyger, tyger burning bright > >> In the forests of the night, > >> What immortal hand or eye > >> Could frame thy fearful symmetry?" > >> > >> William Blake - Songs of Experience -1794 England > >> > > > > > -- -------------------------- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
