"mysql-connector-java" Version for Solr 4.4.0
Hi, Can anyone tell me whats the correct version for "mysql-connector-java" for Solr 4.4.0? I have solr 4.3.0 and use mysql-connector-java-5.1.25-bin.jar. But now I am upgrading to solr 4.4.0. thanks a lot. -- Regards, Raheel Hasan
Re: "mysql-connector-java" Version for Solr 4.4.0
ok thanks... On Tue, Oct 22, 2013 at 2:51 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > You can continue to use the same version of mysql-connector-java. There's > no correct version as such. > > > On Tue, Oct 22, 2013 at 1:35 PM, Raheel Hasan >wrote: > > > Hi, > > > > Can anyone tell me whats the correct version for "mysql-connector-java" > for > > Solr 4.4.0? > > I have solr 4.3.0 and use mysql-connector-java-5.1.25-bin.jar. But now I > am > > upgrading to solr 4.4.0. > > > > thanks a lot. > > > > -- > > Regards, > > Raheel Hasan > > > > > > -- > Regards, > Shalin Shekhar Mangar. > -- Regards, Raheel Hasan
securing Solr Admin
Hi, I want to know how to secure the admin section. The site " http://wiki.apache.org/solr/SolrSecurity"; has a lot of stuff, but I want to put htaccess based restriction. Can anyone tell me where to place the htaccess? I am using solr 4.3 thanks. -- Regards, Raheel Hasan
Re: securing Solr Admin
This sounds like trouble. I have used Solr in my script (php) such that I curl it for query (using "solr/automata/select?q="). If I make it completely off-public, how will my own site access it? Is there any parameter to prevent access by "REMOTE_ADDR"? Thanks. On Tue, Oct 22, 2013 at 6:49 PM, Shawn Heisey wrote: > On 10/22/2013 7:32 AM, Raheel Hasan wrote: > > I want to know how to secure the admin section. The site " > > http://wiki.apache.org/solr/SolrSecurity"; has a lot of stuff, but I > want to > > put htaccess based restriction. > > > > Can anyone tell me where to place the htaccess? > > > > I am using solr 4.3 > > Restricting with htaccess is a webserver function, and it is > specifically an Apache webserver function. I don't think that Servlet > containers use htaccess, but I admit that I'm not very familiar with the > intricacies of servlet configuration. > > Solr itself contains no security features. Security is completely up to > the servlet container. > > A note of caution - certain features of Solr, including SolrCloud and > distributed search, will make requests internally. Adding security > restrictions can break that functionality. > > Solr should not be exposed directly to end users. Queries should be > accepted and sanitized by back-end code before they are sent to Solr, > which should be firewalled so only trusted personnel and applications > can reach it. > > Thanks, > Shawn > > -- Regards, Raheel Hasan
Stop/Restart Solr
Hi, is there a way to stop/restart java? I lost control over it via SSH and connection was closed. But the Solr (start.jar) is still running. thanks. -- Regards, Raheel Hasan
DIH - URLDataSource import size
Hi, I have an issue that is only coming on live environment. The DIH with URLDataSource is not working when the file size imported is large (i.e. 100kb above - which is not so large). If its large, it returns nothing (as seen in the Debug section of DataImport at Solr Admin). However, when working on local environment, this issue doesnt come at all. (note that I am using it with URLDataSource with PlainTextEntityProcessor in the entity field). Please help me as I tried to get it done a lot, but cant !! Thanks a lot. -- Regards, Raheel Hasan
Re: Stop/Restart Solr
ok. I thought there must be a way to restart solr.. Because, once the process is running (i.e. start.jar), I dont know where to find it. pgrep doesnt return start.jar but java as a whole. So how to know the exact process number of start.jar? Thanks.. On Tue, Oct 22, 2013 at 9:50 PM, Yago Riveiro wrote: > If you are asking about if solr has a way to restart himself, I think that > the answer is no. > > If you lost control of the remote machine someone will need to go and > restart the machine ... > > You can try use a kvm or other remote control system > > -- > Yago Riveiro > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte wrote: > > > If you are on linux/unix, use the kill command. > > > > François > > > > On Oct 22, 2013, at 12:42 PM, Raheel Hasan > > raheelhasan@gmail.com)> wrote: > > > > > Hi, > > > > > > is there a way to stop/restart java? I lost control over it via SSH and > > > connection was closed. But the Solr (start.jar) is still running. > > > > > > thanks. > > > > > > -- > > > Regards, > > > Raheel Hasan > > > > > > > > > > > > -- Regards, Raheel Hasan
Re: Stop/Restart Solr
its CentOS... and using jetty with solr here.. On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte < fschietteca...@gmail.com> wrote: > A few more specifics about the environment would help, Windows/Linux/...? > Jetty/Tomcat/...? > > François > > On Oct 22, 2013, at 12:50 PM, Yago Riveiro wrote: > > > If you are asking about if solr has a way to restart himself, I think > that the answer is no. > > > > If you lost control of the remote machine someone will need to go and > restart the machine ... > > > > You can try use a kvm or other remote control system > > > > -- > > Yago Riveiro > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte wrote: > > > >> If you are on linux/unix, use the kill command. > >> > >> François > >> > >> On Oct 22, 2013, at 12:42 PM, Raheel Hasan > >> raheelhasan@gmail.com)> wrote: > >> > >>> Hi, > >>> > >>> is there a way to stop/restart java? I lost control over it via SSH and > >>> connection was closed. But the Solr (start.jar) is still running. > >>> > >>> thanks. > >>> > >>> -- > >>> Regards, > >>> Raheel Hasan > >>> > >> > >> > >> > > > > > > -- Regards, Raheel Hasan
Re: Stop/Restart Solr
ok fantastic... thanks a lot guyz On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte < fschietteca...@gmail.com> wrote: > Yago has the right command to search for the process, that will get you > the process ID specifically the first number on the output line, then do > 'kill ###', if that fails 'kill -9 ###'. > > François > > On Oct 22, 2013, at 12:56 PM, Raheel Hasan > wrote: > > > its CentOS... > > > > and using jetty with solr here.. > > > > > > On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte < > > fschietteca...@gmail.com> wrote: > > > >> A few more specifics about the environment would help, > Windows/Linux/...? > >> Jetty/Tomcat/...? > >> > >> François > >> > >> On Oct 22, 2013, at 12:50 PM, Yago Riveiro > wrote: > >> > >>> If you are asking about if solr has a way to restart himself, I think > >> that the answer is no. > >>> > >>> If you lost control of the remote machine someone will need to go and > >> restart the machine ... > >>> > >>> You can try use a kvm or other remote control system > >>> > >>> -- > >>> Yago Riveiro > >>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > >>> > >>> > >>> On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte wrote: > >>> > >>>> If you are on linux/unix, use the kill command. > >>>> > >>>> François > >>>> > >>>> On Oct 22, 2013, at 12:42 PM, Raheel Hasan (mailto: > >> raheelhasan@gmail.com)> wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> is there a way to stop/restart java? I lost control over it via SSH > and > >>>>> connection was closed. But the Solr (start.jar) is still running. > >>>>> > >>>>> thanks. > >>>>> > >>>>> -- > >>>>> Regards, > >>>>> Raheel Hasan > >>>>> > >>>> > >>>> > >>>> > >>> > >>> > >> > >> > > > > > > -- > > Regards, > > Raheel Hasan > > -- Regards, Raheel Hasan
Re: securing Solr Admin
ok thats very knowledgeable... thanks.. I will try to put a firewall to prevent some access... What I was looking for was some global & simple setting (like in the core setting) that prevents access to certain ip... or an htaccess type settings allowed for the core... But I guess thats not part of solr.. On Tue, Oct 22, 2013 at 10:52 PM, Raymond Wiker wrote: > On Oct 22, 2013, at 19:29 , Shawn Heisey wrote: > > On 10/22/2013 8:09 AM, Raheel Hasan wrote: > >> This sounds like trouble. > >> > >> I have used Solr in my script (php) such that I curl it for query (using > >> "solr/automata/select?q="). If I make it completely off-public, how > will my > >> own site access it? > >> > >> Is there any parameter to prevent access by "REMOTE_ADDR"? > > > > The basic design intent with Solr is this: > > > > 1) A set of servers that handle your website or other applicationthat > uses Solr. > > 2) A set of servers that handle Solr. > > 3) A firewall that allows only relevant traffic from the end users (or > the internet) to reach the webservers on appropriate ports. Only trusted > administrators can reach the Solr servers. If the websites and Solr are not > on the same network, the firewalling should allow the website servers to > talk freely to Solr on Solr's port. > > > > This means that you can't have purely javascript-driven search boxes, > unless the javascript sends the search request to code on the webserver > which cleans it ip and constructs a Solr query from it. > > > > Side issue: There are PHP packages for talking to Solr in an > object-oriented way, rather than using crafting the URLs yourself and using > curl. Here's some examples: > > > > http://pecl.php.net/package/solr > > http://www.solarium-project.org/ > > > > I don't write PHP code myself, but it is usually a lot easier to deal > with a Solr API than making URLs yourself and parsing the responses. > > > > Thanks, > > Shawn > > > > I have numerous search applications that only involve SOLR, jQuery, > Apache... and two additional server processes, one of which does query > validation, adds filtering and does an XSL transform of the search results, > while the other does a number of application-specific support tasks. No > SOLR API, except for HTTP. > > The most recent search interface I did uses "solrstrap" to do most of the > UI work. > > In summary: No problem making a search interface that runs as a single web > page. > > > -- Regards, Raheel Hasan
Re: DIH - URLDataSource import size
anyone? On Tue, Oct 22, 2013 at 9:50 PM, Raheel Hasan wrote: > Hi, > > I have an issue that is only coming on live environment. The DIH > with URLDataSource is not working when the file size imported is large > (i.e. 100kb above - which is not so large). If its large, it returns > nothing (as seen in the Debug section of DataImport at Solr Admin). > > However, when working on local environment, this issue doesnt come at all. > > (note that I am using it with URLDataSource with PlainTextEntityProcessor > in the entity field). > > Please help me as I tried to get it done a lot, but cant !! > > Thanks a lot. > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Issue with large html indexing
Hi, I have an issue here while indexing large html. Here is the confguration for that: 1) Data is imported via URLDataSource / PlainTextEntityProcessor (DIH) 2) Schema has this for the field: type="text_en_splitting" indexed="true" stored="false" required="false" 3) text_en_splitting has the following work done for indexing: HTMLStripCharFilterFactory WhitespaceTokenizerFactory (create tokens) StopFilterFactory WordDelimiterFilterFactory ICUFoldingFilterFactory PorterStemFilterFactory RemoveDuplicatesTokenFilterFactory LengthFilterFactory However, the indexed data is like this (as in the attached image): [image: Inline image 1] so what are these numbers? If I put small html, it works fine, but as the size of html file increases, this is what happens.. -- Regards, Raheel Hasan
Re: Stop/Restart Solr
Kill -9 didnt kill it... ... the process is now again listed, but with PPID=1 which I dont want to kill as many processes have this same id... On Tue, Oct 22, 2013 at 11:59 PM, Utkarsh Sengar wrote: > We use this to start/stop solr: > > Start: > java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore > -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 > -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar > > Stop: > java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 > -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar --stop > > > Thanks, > -Utkarsh > > > > On Tue, Oct 22, 2013 at 10:09 AM, Raheel Hasan >wrote: > > > ok fantastic... thanks a lot guyz > > > > > > On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte < > > fschietteca...@gmail.com> wrote: > > > > > Yago has the right command to search for the process, that will get you > > > the process ID specifically the first number on the output line, then > do > > > 'kill ###', if that fails 'kill -9 ###'. > > > > > > François > > > > > > On Oct 22, 2013, at 12:56 PM, Raheel Hasan > > > wrote: > > > > > > > its CentOS... > > > > > > > > and using jetty with solr here.. > > > > > > > > > > > > On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte < > > > > fschietteca...@gmail.com> wrote: > > > > > > > >> A few more specifics about the environment would help, > > > Windows/Linux/...? > > > >> Jetty/Tomcat/...? > > > >> > > > >> François > > > >> > > > >> On Oct 22, 2013, at 12:50 PM, Yago Riveiro > > > wrote: > > > >> > > > >>> If you are asking about if solr has a way to restart himself, I > think > > > >> that the answer is no. > > > >>> > > > >>> If you lost control of the remote machine someone will need to go > and > > > >> restart the machine ... > > > >>> > > > >>> You can try use a kvm or other remote control system > > > >>> > > > >>> -- > > > >>> Yago Riveiro > > > >>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > >>> > > > >>> > > > >>> On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte > > wrote: > > > >>> > > > >>>> If you are on linux/unix, use the kill command. > > > >>>> > > > >>>> François > > > >>>> > > > >>>> On Oct 22, 2013, at 12:42 PM, Raheel Hasan < > > raheelhasan@gmail.com > > > (mailto: > > > >> raheelhasan@gmail.com)> wrote: > > > >>>> > > > >>>>> Hi, > > > >>>>> > > > >>>>> is there a way to stop/restart java? I lost control over it via > SSH > > > and > > > >>>>> connection was closed. But the Solr (start.jar) is still running. > > > >>>>> > > > >>>>> thanks. > > > >>>>> > > > >>>>> -- > > > >>>>> Regards, > > > >>>>> Raheel Hasan > > > >>>>> > > > >>>> > > > >>>> > > > >>>> > > > >>> > > > >>> > > > >> > > > >> > > > > > > > > > > > > -- > > > > Regards, > > > > Raheel Hasan > > > > > > > > > > > > -- > > Regards, > > Raheel Hasan > > > > > > -- > Thanks, > -Utkarsh > -- Regards, Raheel Hasan
Re: Stop/Restart Solr
also, is this DSTOP.PORT same as on which solr is visible on a browser (i.e. like 8983 from http://localhost:8983)? On Wed, Oct 23, 2013 at 2:49 PM, Raheel Hasan wrote: > Kill -9 didnt kill it... ... the process is now again listed, but > with PPID=1 which I dont want to kill as many processes have this same id... > > > On Tue, Oct 22, 2013 at 11:59 PM, Utkarsh Sengar > wrote: > >> We use this to start/stop solr: >> >> Start: >> java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore >> -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 >> -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar >> >> Stop: >> java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 >> -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar --stop >> >> >> Thanks, >> -Utkarsh >> >> >> >> On Tue, Oct 22, 2013 at 10:09 AM, Raheel Hasan > >wrote: >> >> > ok fantastic... thanks a lot guyz >> > >> > >> > On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte < >> > fschietteca...@gmail.com> wrote: >> > >> > > Yago has the right command to search for the process, that will get >> you >> > > the process ID specifically the first number on the output line, then >> do >> > > 'kill ###', if that fails 'kill -9 ###'. >> > > >> > > François >> > > >> > > On Oct 22, 2013, at 12:56 PM, Raheel Hasan > > >> > > wrote: >> > > >> > > > its CentOS... >> > > > >> > > > and using jetty with solr here.. >> > > > >> > > > >> > > > On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte < >> > > > fschietteca...@gmail.com> wrote: >> > > > >> > > >> A few more specifics about the environment would help, >> > > Windows/Linux/...? >> > > >> Jetty/Tomcat/...? >> > > >> >> > > >> François >> > > >> >> > > >> On Oct 22, 2013, at 12:50 PM, Yago Riveiro > > >> > > wrote: >> > > >> >> > > >>> If you are asking about if solr has a way to restart himself, I >> think >> > > >> that the answer is no. >> > > >>> >> > > >>> If you lost control of the remote machine someone will need to go >> and >> > > >> restart the machine ... >> > > >>> >> > > >>> You can try use a kvm or other remote control system >> > > >>> >> > > >>> -- >> > > >>> Yago Riveiro >> > > >>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig) >> > > >>> >> > > >>> >> > > >>> On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte >> > wrote: >> > > >>> >> > > >>>> If you are on linux/unix, use the kill command. >> > > >>>> >> > > >>>> François >> > > >>>> >> > > >>>> On Oct 22, 2013, at 12:42 PM, Raheel Hasan < >> > raheelhasan@gmail.com >> > > (mailto: >> > > >> raheelhasan@gmail.com)> wrote: >> > > >>>> >> > > >>>>> Hi, >> > > >>>>> >> > > >>>>> is there a way to stop/restart java? I lost control over it via >> SSH >> > > and >> > > >>>>> connection was closed. But the Solr (start.jar) is still >> running. >> > > >>>>> >> > > >>>>> thanks. >> > > >>>>> >> > > >>>>> -- >> > > >>>>> Regards, >> > > >>>>> Raheel Hasan >> > > >>>>> >> > > >>>> >> > > >>>> >> > > >>>> >> > > >>> >> > > >>> >> > > >> >> > > >> >> > > > >> > > > >> > > > -- >> > > > Regards, >> > > > Raheel Hasan >> > > >> > > >> > >> > >> > -- >> > Regards, >> > Raheel Hasan >> > >> >> >> >> -- >> Thanks, >> -Utkarsh >> > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Stop/Restart Solr
31173 1 0 16:45 ?00:00:08 java -jar start.jar On Wed, Oct 23, 2013 at 2:53 PM, Jeevanandam M. wrote: > Can you please share output of following command? > ps -ef | grep 'start.jar' > > - Jeeva > > -- Original Message ------ > From: Raheel Hasan [mailto:raheelhasan@gmail.com] > Sent: October 23, 2013 3:19:46 PM GMT+05:30 > To: solr-user@lucene.apache.org > Subject: Re: Stop/Restart Solr > > > Kill -9 didnt kill it... ... the process is now again listed, but with > PPID=1 which I dont want to kill as many processes have this same id... > > > On Tue, Oct 22, 2013 at 11:59 PM, Utkarsh Sengar >wrote: > > > We use this to start/stop solr: > > > > Start: > > java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore > > -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 > > -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar > > > > Stop: > > java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 > > -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar --stop > > > > > > Thanks, > > -Utkarsh > > > > > > > > On Tue, Oct 22, 2013 at 10:09 AM, Raheel Hasan < > raheelhasan@gmail.com > >> wrote: > > > >> ok fantastic... thanks a lot guyz > >> > >> > >> On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte < > >> fschietteca...@gmail.com> wrote: > >> > >>> Yago has the right command to search for the process, that will get you > >>> the process ID specifically the first number on the output line, then > > do > >>> 'kill ###', if that fails 'kill -9 ###'. > >>> > >>> François > >>> > >>> On Oct 22, 2013, at 12:56 PM, Raheel Hasan > >>> wrote: > >>> > >>>> its CentOS... > >>>> > >>>> and using jetty with solr here.. > >>>> > >>>> > >>>> On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte < > >>>> fschietteca...@gmail.com> wrote: > >>>> > >>>>> A few more specifics about the environment would help, > >>> Windows/Linux/...? > >>>>> Jetty/Tomcat/...? > >>>>> > >>>>> François > >>>>> > >>>>> On Oct 22, 2013, at 12:50 PM, Yago Riveiro > >>> wrote: > >>>>> > >>>>>> If you are asking about if solr has a way to restart himself, I > > think > >>>>> that the answer is no. > >>>>>> > >>>>>> If you lost control of the remote machine someone will need to go > > and > >>>>> restart the machine ... > >>>>>> > >>>>>> You can try use a kvm or other remote control system > >>>>>> > >>>>>> -- > >>>>>> Yago Riveiro > >>>>>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > >>>>>> > >>>>>> > >>>>>> On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte > >> wrote: > >>>>>> > >>>>>>> If you are on linux/unix, use the kill command. > >>>>>>> > >>>>>>> François > >>>>>>> > >>>>>>> On Oct 22, 2013, at 12:42 PM, Raheel Hasan < > >> raheelhasan@gmail.com > >>> (mailto: > >>>>> raheelhasan@gmail.com)> wrote: > >>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> is there a way to stop/restart java? I lost control over it via > > SSH > >>> and > >>>>>>>> connection was closed. But the Solr (start.jar) is still running. > >>>>>>>> > >>>>>>>> thanks. > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Regards, > >>>>>>>> Raheel Hasan > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Regards, > >>>> Raheel Hasan > >>> > >>> > >> > >> > >> -- > >> Regards, > >> Raheel Hasan > >> > > > > > > > > -- > > Thanks, > > -Utkarsh > > > > > > -- > Regards, > Raheel Hasan > > > > -- Regards, Raheel Hasan
Re: Stop/Restart Solr
ok got it thanks :) On Wed, Oct 23, 2013 at 7:33 PM, Walter Underwood wrote: > PPID is the "parent process ID". You want to kill the PID, not the PPID. > > wunder > > On Oct 23, 2013, at 3:09 AM, Jeevanandam M. wrote: > > > It seems process started recently. Is there any external cron/process > triggering a startup of Solr? > > Kill again and monitor it. > > > > - Jeeva > > > > -- Original Message -- > > From: Raheel Hasan [mailto:raheelhasan@gmail.com] > > Sent: October 23, 2013 3:29:47 PM GMT+05:30 > > To: solr-user@lucene.apache.org > > Subject: Re: Stop/Restart Solr > > > > > > 31173 1 0 16:45 ?00:00:08 java -jar start.jar > > > > > > On Wed, Oct 23, 2013 at 2:53 PM, Jeevanandam M. > wrote: > > > >> Can you please share output of following command? > >> ps -ef | grep 'start.jar' > >> > >> - Jeeva > >> > >> -- Original Message -- > >> From: Raheel Hasan [mailto:raheelhasan@gmail.com] > >> Sent: October 23, 2013 3:19:46 PM GMT+05:30 > >> To: solr-user@lucene.apache.org > >> Subject: Re: Stop/Restart Solr > >> > >> > >> Kill -9 didnt kill it... ... the process is now again listed, but > with > >> PPID=1 which I dont want to kill as many processes have this same id... > >> > >> > >> On Tue, Oct 22, 2013 at 11:59 PM, Utkarsh Sengar >>> wrote: > >> > >>> We use this to start/stop solr: > >>> > >>> Start: > >>> java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore > >>> -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 > >>> -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar > >>> > >>> Stop: > >>> java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 > >>> -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar --stop > >>> > >>> > >>> Thanks, > >>> -Utkarsh > >>> > >>> > >>> > >>> On Tue, Oct 22, 2013 at 10:09 AM, Raheel Hasan < > >> raheelhasan@gmail.com > >>>> wrote: > >>> > >>>> ok fantastic... thanks a lot guyz > >>>> > >>>> > >>>> On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte < > >>>> fschietteca...@gmail.com> wrote: > >>>> > >>>>> Yago has the right command to search for the process, that will get > you > >>>>> the process ID specifically the first number on the output line, then > >>> do > >>>>> 'kill ###', if that fails 'kill -9 ###'. > >>>>> > >>>>> François > >>>>> > >>>>> On Oct 22, 2013, at 12:56 PM, Raheel Hasan < > raheelhasan@gmail.com> > >>>>> wrote: > >>>>> > >>>>>> its CentOS... > >>>>>> > >>>>>> and using jetty with solr here.. > >>>>>> > >>>>>> > >>>>>> On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte < > >>>>>> fschietteca...@gmail.com> wrote: > >>>>>> > >>>>>>> A few more specifics about the environment would help, > >>>>> Windows/Linux/...? > >>>>>>> Jetty/Tomcat/...? > >>>>>>> > >>>>>>> François > >>>>>>> > >>>>>>> On Oct 22, 2013, at 12:50 PM, Yago Riveiro > > >>>>> wrote: > >>>>>>> > >>>>>>>> If you are asking about if solr has a way to restart himself, I > >>> think > >>>>>>> that the answer is no. > >>>>>>>> > >>>>>>>> If you lost control of the remote machine someone will need to go > >>> and > >>>>>>> restart the machine ... > >>>>>>>> > >>>>>>>> You can try use a kvm or other remote control system > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Yago Riveiro > >>>>>>>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > >>>>>>>> > >>>>>>>> > >>>>>>>> On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte > >>>> wrote: > >>>>>>>> > >>>>>>>>> If you are on linux/unix, use the kill command. > >>>>>>>>> > >>>>>>>>> François > >>>>>>>>> > >>>>>>>>> On Oct 22, 2013, at 12:42 PM, Raheel Hasan < > >>>> raheelhasan@gmail.com > >>>>> (mailto: > >>>>>>> raheelhasan@gmail.com)> wrote: > >>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> is there a way to stop/restart java? I lost control over it via > >>> SSH > >>>>> and > >>>>>>>>>> connection was closed. But the Solr (start.jar) is still > running. > >>>>>>>>>> > >>>>>>>>>> thanks. > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Regards, > >>>>>>>>>> Raheel Hasan > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Regards, > >>>>>> Raheel Hasan > >>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Regards, > >>>> Raheel Hasan > >>>> > >>> > >>> > >>> > >>> -- > >>> Thanks, > >>> -Utkarsh > >>> > >> > >> > >> > >> -- > >> Regards, > >> Raheel Hasan > >> > >> > >> > >> > > > > > > -- > > Regards, > > Raheel Hasan > > > > > > > > -- > Walter Underwood > wun...@wunderwood.org > > > > -- Regards, Raheel Hasan
Re: Issue with large html indexing
ok. see this: http://s23.postimg.org/yck2s5k1n/html_indexing.png On Wed, Oct 23, 2013 at 10:45 PM, Erick Erickson wrote: > Attachments and images are often eaten by the mail server, your image is > not visible at least to me. Can you describe what you're seeing? Or post > the image somewhere and provide a link? > > Best, > Erick > > > On Wed, Oct 23, 2013 at 11:07 AM, Raheel Hasan >wrote: > > > Hi, > > > > I have an issue here while indexing large html. Here is the confguration > > for that: > > > > 1) Data is imported via URLDataSource / PlainTextEntityProcessor (DIH) > > > > 2) Schema has this for the field: > > type="text_en_splitting" indexed="true" stored="false" required="false" > > > > 3) text_en_splitting has the following work done for indexing: > > HTMLStripCharFilterFactory > > WhitespaceTokenizerFactory (create tokens) > > StopFilterFactory > > WordDelimiterFilterFactory > > ICUFoldingFilterFactory > > PorterStemFilterFactory > > RemoveDuplicatesTokenFilterFactory > > LengthFilterFactory > > > > However, the indexed data is like this (as in the attached image): > > [image: Inline image 1] > > > > > > so what are these numbers? > > If I put small html, it works fine, but as the size of html file > > increases, this is what happens.. > > > > -- > > Regards, > > Raheel Hasan > > > -- Regards, Raheel Hasan
Solr 4.3.1 :: Error loading class 'solr.ICUFoldingFilterFactory'
Hi,
I got a new issue now. I have Solr 4.3.0 running just fine. However on Solr
4.3.1, it wont load. I get this issue:
{msg=SolrCore 'mycore' is not available due to init failure: Plugin
init failure for [schema.xml] fieldType "text_ws": Plugin init failure
for [schema.xml] analyzer/filter: Error loading class
'solr.ICUFoldingFilterFactory',trace=org.apache.solr.common.SolrException:
SolrCore 'mycore' is not available due to init failure: Plugin init
failure for [schema.xml] fieldType "text_ws": Plugin init failure for
[schema.xml] analyzer/filter: Error loading class
'solr.ICUFoldingFilterFactory'
Here is Solr.xml
--
Regards,
Raheel Hasan
Re: Solr 4.3.1 :: Error loading class 'solr.ICUFoldingFilterFactory'
Thanks Shawn. Always grateful for your help...
On Wed, Nov 27, 2013 at 10:37 PM, Shawn Heisey wrote:
> On 11/27/2013 9:37 AM, Raheel Hasan wrote:
>
>> I got a new issue now. I have Solr 4.3.0 running just fine. However on
>> Solr
>> 4.3.1, it wont load. I get this issue:
>>
>>
>> {msg=SolrCore 'mycore' is not available due to init failure: Plugin
>> init failure for [schema.xml] fieldType "text_ws": Plugin init failure
>> for [schema.xml] analyzer/filter: Error loading class
>> 'solr.ICUFoldingFilterFactory',trace=org.apache.solr.common.
>> SolrException:
>> SolrCore 'mycore' is not available due to init failure: Plugin init
>> failure for [schema.xml] fieldType "text_ws": Plugin init failure for
>> [schema.xml] analyzer/filter: Error loading class
>> 'solr.ICUFoldingFilterFactory'
>>
>
> The jars required for that analysis chain component are not available to
> Solr. Jars can be loaded in one of two ways. 1) By using lib directives
> in solrconfig.xml. 2) Putting them all in ${solr.solr.home}/lib, with
> ${solr.solr.home} as the location where solr.xml lives. The latter is a
> far better option.Since you are using 4.3.1, don't use the sharedLib
> attribute in solr.xml, or you'll run into SOLR-4852.
>
> The extra jars required for ICUFoldingFilterFactory on Solr 4.3.1 are:
>
> icu4j-49.1.jar
> lucene-analyzers-icu-4.3.1.jar
>
> You can find these in the download under contrib/analysis-extras.
>
> Thanks,
> Shawn
>
>
--
Regards,
Raheel Hasan
Data Import from RDBMS+File
Hi everyone, I am looking for a way to import/index data such that i load data from table_1 and instead of joining from table_2, i import the rest of the "joined" data from a file instead. The name of the file comes from a field from table_1 Is it possible? and is it easily possible? -- Regards, Raheel Hasan
Re: Data Import from RDBMS+File
On this page (http://wiki.apache.org/solr/DataImportHandler), I cant see how its possible. Perhaps there is another guide.. Basically, this is what I am doing: Index data from multiple tables into Solr (see here http://wiki.apache.org/solr/DIHQuickStart). I need to skip 1 very big heavy table as it only have 1 field that is a complete file. So I want to skip the step of loading that file per record into my RDB and then indexing it... Instead, I want to directly index that file with the rest of the records from coming from database... On Mon, Jul 8, 2013 at 7:30 PM, Alexandre Rafalovitch wrote: > Did you have a chance to look at DIH with nested entities yet? That's > probably the way to go to start out. > > Or a custom client, of course. Or, ETL solutions that support Solr (e.g. > Apache Flume - not personally tested yet). > > Regards, >Alex. > > Personal website: http://www.outerthoughts.com/ > LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch > - Time is the quality of nature that keeps events from happening all at > once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) > > > On Mon, Jul 8, 2013 at 10:08 AM, Raheel Hasan >wrote: > > > Hi everyone, > > > > I am looking for a way to import/index data such that i load data from > > table_1 and instead of joining from table_2, i import the rest of the > > "joined" data from a file instead. The name of the file comes from a > field > > from table_1 > > > > Is it possible? and is it easily possible? > > > > -- > > Regards, > > Raheel Hasan > > > -- Regards, Raheel Hasan
Re: Data Import from RDBMS+File
ok great.
can I use this EntityProcessor within JdbcDataSource?
Like this:
http://localhost/project_1/files/a.txt";
dataSource="data-source-name">
By the way, I currently load the field into "text_en_splitting" as defined
in schema.xml...
On Mon, Jul 8, 2013 at 7:59 PM, Alexandre Rafalovitch wrote:
> http://wiki.apache.org/solr/DataImportHandler#PlainTextEntityProcessor or
> http://wiki.apache.org/solr/DataImportHandler#LineEntityProcessor ?
>
> The file name gets exposed as a ${entityname.fieldname} variable. You can
> probably copy/manipulate it with a transformer on the external entity
> before it hits an inner one.
>
> Regards,
> Alex.
>
> Personal website: http://www.outerthoughts.com/
> LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> - Time is the quality of nature that keeps events from happening all at
> once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
>
>
> On Mon, Jul 8, 2013 at 10:42 AM, Raheel Hasan >wrote:
>
> > On this page (http://wiki.apache.org/solr/DataImportHandler), I cant see
> > how its possible. Perhaps there is another guide..
> >
> > Basically, this is what I am doing:
> > Index data from multiple tables into Solr (see here
> > http://wiki.apache.org/solr/DIHQuickStart). I need to skip 1 very big
> > heavy
> > table as it only have 1 field that is a complete file. So I want to skip
> > the step of loading that file per record into my RDB and then indexing
> > it... Instead, I want to directly index that file with the rest of the
> > records from coming from database...
> >
> >
> >
> >
> > On Mon, Jul 8, 2013 at 7:30 PM, Alexandre Rafalovitch <
> arafa...@gmail.com
> > >wrote:
> >
> > > Did you have a chance to look at DIH with nested entities yet? That's
> > > probably the way to go to start out.
> > >
> > > Or a custom client, of course. Or, ETL solutions that support Solr
> (e.g.
> > > Apache Flume - not personally tested yet).
> > >
> > > Regards,
> > >Alex.
> > >
> > > Personal website: http://www.outerthoughts.com/
> > > LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> > > - Time is the quality of nature that keeps events from happening all at
> > > once. Lately, it doesn't seem to be working. (Anonymous - via GTD
> book)
> > >
> > >
> > > On Mon, Jul 8, 2013 at 10:08 AM, Raheel Hasan <
> raheelhasan@gmail.com
> > > >wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > I am looking for a way to import/index data such that i load data
> from
> > > > table_1 and instead of joining from table_2, i import the rest of the
> > > > "joined" data from a file instead. The name of the file comes from a
> > > field
> > > > from table_1
> > > >
> > > > Is it possible? and is it easily possible?
> > > >
> > > > --
> > > > Regards,
> > > > Raheel Hasan
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Raheel Hasan
> >
>
--
Regards,
Raheel Hasan
Facet Count and RegexTransformer>splitBy
Hi guyz, Just a quick question: I have a field that has CSV values in the database. So I will use the DataImportHandler and will index it using RegexTransformer's splitBy attribute. However, since this is the first time I am doing it, I just wanted to be sure if it will work for Facet Count? For example: >From "query" results (say this is the values in that field): row 1 = 1,2,3,4 row 2 = 1,4,5,3 row 3 = 2,1,20,66 . . . . so facet count will get me: '1' = 3 occurrence '2' = 2 occur. . . .and so on. -- Regards, Raheel Hasan
Re: Facet Count and RegexTransformer>splitBy
Its a csv from the database. I will import it like this, (say for example the field is 'emailids' and it contain csv of email ids): On Fri, Sep 6, 2013 at 9:01 PM, Jack Krupansky wrote: > You're not being clear here - are the commas delimiting fields or do you > have one value per row? > > Yes, you can tokenize a comma-delimited value in Solr. > > > -- Jack Krupansky > > -----Original Message- From: Raheel Hasan > Sent: Friday, September 06, 2013 11:54 AM > To: solr-user@lucene.apache.org > Subject: Re: Facet Count and RegexTransformer>splitBy > > > Hi, > > What I want is very simple: > > The "query" results: > row 1 = a,b,c,d > row 2 = a,f,r,e > row 3 = a,c,ff,e,b > .. > > facet count needed: > 'a' = 3 occurrence > 'b' = 2 occur. > 'c' = 2 occur. > . > . > . > > > I searched and found a solution here: > http://stackoverflow.com/**questions/9914483/solr-facet-** > multiple-words-with-comma-**separated-values<http://stackoverflow.com/questions/9914483/solr-facet-multiple-words-with-comma-separated-values> > > But I want to be sure if it will work. > > > > On Fri, Sep 6, 2013 at 8:20 PM, Jack Krupansky ** > wrote: > > Facet counts are per field - your counts are scattered across different >> fields. >> >> There are additional capabilities in the facet component, but first you >> should describe exactly what your requirements are. >> >> -- Jack Krupansky >> -Original Message- From: Raheel Hasan >> Sent: Friday, September 06, 2013 9:58 AM >> To: solr-user@lucene.apache.org >> Subject: Facet Count and RegexTransformer>splitBy >> >> >> Hi guyz, >> >> Just a quick question: >> >> I have a field that has CSV values in the database. So I will use the >> DataImportHandler and will index it using RegexTransformer's splitBy >> attribute. However, since this is the first time I am doing it, I just >> wanted to be sure if it will work for Facet Count? >> >> For example: >> From "query" results (say this is the values in that field): >> row 1 = 1,2,3,4 >> row 2 = 1,4,5,3 >> row 3 = 2,1,20,66 >> . >> . >> . >> . >> so facet count will get me: >> '1' = 3 occurrence >> '2' = 2 occur. >> . >> . >> .and so on. >> >> >> >> >> >> -- >> Regards, >> Raheel Hasan >> >> > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Facet Count and RegexTransformer>splitBy
let me further elaborate: [db>table1] field1 = int field2= string (solr indexing = true) field3 = csv [During import into solr] splitBy="," [After import] solr will be searched for terms from field2. [needed] counts of occurrances of each value in csv On Fri, Sep 6, 2013 at 9:35 PM, Raheel Hasan wrote: > Its a csv from the database. I will import it like this, (say for example > the field is 'emailids' and it contain csv of email ids): > > > > > On Fri, Sep 6, 2013 at 9:01 PM, Jack Krupansky wrote: > >> You're not being clear here - are the commas delimiting fields or do you >> have one value per row? >> >> Yes, you can tokenize a comma-delimited value in Solr. >> >> >> -- Jack Krupansky >> >> -Original Message- From: Raheel Hasan >> Sent: Friday, September 06, 2013 11:54 AM >> To: solr-user@lucene.apache.org >> Subject: Re: Facet Count and RegexTransformer>splitBy >> >> >> Hi, >> >> What I want is very simple: >> >> The "query" results: >> row 1 = a,b,c,d >> row 2 = a,f,r,e >> row 3 = a,c,ff,e,b >> .. >> >> facet count needed: >> 'a' = 3 occurrence >> 'b' = 2 occur. >> 'c' = 2 occur. >> . >> . >> . >> >> >> I searched and found a solution here: >> http://stackoverflow.com/**questions/9914483/solr-facet-** >> multiple-words-with-comma-**separated-values<http://stackoverflow.com/questions/9914483/solr-facet-multiple-words-with-comma-separated-values> >> >> But I want to be sure if it will work. >> >> >> >> On Fri, Sep 6, 2013 at 8:20 PM, Jack Krupansky * >> *wrote: >> >> Facet counts are per field - your counts are scattered across different >>> fields. >>> >>> There are additional capabilities in the facet component, but first you >>> should describe exactly what your requirements are. >>> >>> -- Jack Krupansky >>> -Original Message- From: Raheel Hasan >>> Sent: Friday, September 06, 2013 9:58 AM >>> To: solr-user@lucene.apache.org >>> Subject: Facet Count and RegexTransformer>splitBy >>> >>> >>> Hi guyz, >>> >>> Just a quick question: >>> >>> I have a field that has CSV values in the database. So I will use the >>> DataImportHandler and will index it using RegexTransformer's splitBy >>> attribute. However, since this is the first time I am doing it, I just >>> wanted to be sure if it will work for Facet Count? >>> >>> For example: >>> From "query" results (say this is the values in that field): >>> row 1 = 1,2,3,4 >>> row 2 = 1,4,5,3 >>> row 3 = 2,1,20,66 >>> . >>> . >>> . >>> . >>> so facet count will get me: >>> '1' = 3 occurrence >>> '2' = 2 occur. >>> . >>> . >>> .and so on. >>> >>> >>> >>> >>> >>> -- >>> Regards, >>> Raheel Hasan >>> >>> >> >> >> -- >> Regards, >> Raheel Hasan >> > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Facet Count and RegexTransformer>splitBy
basically, a field having a csv... and find counts / number of occurrance of each csv value.. On Fri, Sep 6, 2013 at 8:54 PM, Raheel Hasan wrote: > Hi, > > What I want is very simple: > > The "query" results: > row 1 = a,b,c,d > row 2 = a,f,r,e > row 3 = a,c,ff,e,b > .. > > facet count needed: > 'a' = 3 occurrence > 'b' = 2 occur. > 'c' = 2 occur. > . > . > . > > > I searched and found a solution here: > > http://stackoverflow.com/questions/9914483/solr-facet-multiple-words-with-comma-separated-values > > But I want to be sure if it will work. > > > > On Fri, Sep 6, 2013 at 8:20 PM, Jack Krupansky wrote: > >> Facet counts are per field - your counts are scattered across different >> fields. >> >> There are additional capabilities in the facet component, but first you >> should describe exactly what your requirements are. >> >> -- Jack Krupansky >> -Original Message- From: Raheel Hasan >> Sent: Friday, September 06, 2013 9:58 AM >> To: solr-user@lucene.apache.org >> Subject: Facet Count and RegexTransformer>splitBy >> >> >> Hi guyz, >> >> Just a quick question: >> >> I have a field that has CSV values in the database. So I will use the >> DataImportHandler and will index it using RegexTransformer's splitBy >> attribute. However, since this is the first time I am doing it, I just >> wanted to be sure if it will work for Facet Count? >> >> For example: >> From "query" results (say this is the values in that field): >> row 1 = 1,2,3,4 >> row 2 = 1,4,5,3 >> row 3 = 2,1,20,66 >> . >> . >> . >> . >> so facet count will get me: >> '1' = 3 occurrence >> '2' = 2 occur. >> . >> . >> .and so on. >> >> >> >> >> >> -- >> Regards, >> Raheel Hasan >> > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Facet Count and RegexTransformer>splitBy
Hi, What I want is very simple: The "query" results: row 1 = a,b,c,d row 2 = a,f,r,e row 3 = a,c,ff,e,b .. facet count needed: 'a' = 3 occurrence 'b' = 2 occur. 'c' = 2 occur. . . . I searched and found a solution here: http://stackoverflow.com/questions/9914483/solr-facet-multiple-words-with-comma-separated-values But I want to be sure if it will work. On Fri, Sep 6, 2013 at 8:20 PM, Jack Krupansky wrote: > Facet counts are per field - your counts are scattered across different > fields. > > There are additional capabilities in the facet component, but first you > should describe exactly what your requirements are. > > -- Jack Krupansky > -Original Message- From: Raheel Hasan > Sent: Friday, September 06, 2013 9:58 AM > To: solr-user@lucene.apache.org > Subject: Facet Count and RegexTransformer>splitBy > > > Hi guyz, > > Just a quick question: > > I have a field that has CSV values in the database. So I will use the > DataImportHandler and will index it using RegexTransformer's splitBy > attribute. However, since this is the first time I am doing it, I just > wanted to be sure if it will work for Facet Count? > > For example: > From "query" results (say this is the values in that field): > row 1 = 1,2,3,4 > row 2 = 1,4,5,3 > row 3 = 2,1,20,66 > . > . > . > . > so facet count will get me: > '1' = 3 occurrence > '2' = 2 occur. > . > . > .and so on. > > > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Ideal Server Environment
Hi guyz, I am trying to setup a LIVE environment for my project that uses Apache Solr along with PHP/MySQL... The indexing is of heavy data (about many GBs).. Please can someone recommend the best server for this? Thanks a lot. -- Regards, Raheel Hasan
Re: Ideal Server Environment
Also, I wonder if Solr will require High processor? High Memory or High Storage? 1) For Indexing 2) For querying On Mon, Sep 9, 2013 at 12:36 PM, Raheel Hasan wrote: > Hi guyz, > > I am trying to setup a LIVE environment for my project that uses Apache > Solr along with PHP/MySQL... > > The indexing is of heavy data (about many GBs).. > > Please can someone recommend the best server for this? > > Thanks a lot. > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Ideal Server Environment
ok thanks for the reply Also, could you tell me if CentOS or Ubuntu will be better? On Mon, Sep 9, 2013 at 3:17 PM, Toke Eskildsen wrote: > On Mon, 2013-09-09 at 09:39 +0200, Raheel Hasan wrote: > > Also, I wonder if Solr will require High processor? High Memory or High > > Storage? > > > > 1) For Indexing > > * Processor > * Bulk read/write. > > > 2) For querying > > * Processor only if you have complex queries > * Fast random I/O reads, which can be accomplished either by having > enough RAM to cache most or all of your index or by using SSDs. > > > Your question is much too generic to go into specific hardware. Read > https://wiki.apache.org/lucene-java/ImproveIndexingSpeed > https://wiki.apache.org/lucene-java/ImproveSearchingSpeed > https://wiki.apache.org/solr/SolrPerformanceProblems > then build a test instance, measure and scale from there. > > - Toke Eskildsen > > -- Regards, Raheel Hasan
faceting from multiValued field
Hi, I am having a problem with multiValued field and Faceting This is the schema: all I get is: Note: the data is correctly placed in the field as the query results shows. However, the facet is not working. Could anyone tell me how to achieve it? Thanks a lot. -- Regards, Raheel Hasan
Re: faceting from multiValued field
wow, that did it... fantastic... Thanks a lot :) On Tue, Sep 10, 2013 at 3:56 PM, Rafał Kuć wrote: > Hello! > > Your field needs to be indexed in order for faceting to work. > > -- > Regards, > Rafał Kuć > Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch > > > Hi, > > > I am having a problem with multiValued field and Faceting > > > This is the schema: > > > required="false" omitTermFreqAndPositions="true" multiValued="true" /> > > > all I get is: > > > > > > Note: the data is correctly placed in the field as the query results > shows. > > However, the facet is not working. > > > Could anyone tell me how to achieve it? > > > Thanks a lot. > > > -- Regards, Raheel Hasan
URLDataSource & PlainTextEntityProcessor not working
Hi, I am trying to load data (as plaint text) from a URL. For this I am using URLDataSource & PlainTextEntityProcesso. However, I have the following not working. I checked access logs of my web server, the url is not even getting called: http://localhost/update_1/test.txt";> Can anyone tell me whats the issue here? -- Regards, Raheel Hasan
Re: How to facet data from a multivalued field?
oh got it.. Thanks a lot... On Tue, Sep 10, 2013 at 10:10 PM, Erick Erickson wrote: > You can't facet on fields where indexed="false". When you look at > output docs, you're seeing _stored_ not indexed data. Set > indexed="true" and re-index... > > Best, > Erick > > > On Tue, Sep 10, 2013 at 5:51 AM, Rah1x wrote: > > > Hi buddy, > > > > I am having this problem that I cant even reach to what you did at first > > step.. > > > > all I get is: > > > > > > This is the schema: > > > required="false" omitTermFreqAndPositions="true" multiValued="true" /> > > > > Note: the data is correctly placed in the field as the query results > shows. > > However, the facet is not working. > > > > Could you please share the schema of what you did to achieve it? > > > > Thanks a lot. > > > > > > > > -- > > View this message in context: > > > http://lucene.472066.n3.nabble.com/How-to-facet-data-from-a-multivalued-field-tp3897853p4089045.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > -- Regards, Raheel Hasan
Facet counting empty as well.. how to prevent this?
Hi, I got a small issue here, my facet settings are returning counts for empty "". I.e. when no the actual field was empty. Here are the facet settings: count 6 1 false and this is the part of the result I dont want: 4 (that is coming because the query results had 4 rows with no value in that field whole facet counts are being called). Rest all is working just fine -- Regards, Raheel Hasan
Re: Facet counting empty as well.. how to prevent this?
ok, so I got the idea... I will pull 7 fields instead and remove the empty one... But there must be some setting that can be done in Facet configuration to ignore certain value if we want to On Thu, Sep 12, 2013 at 7:44 PM, Shawn Heisey wrote: > On 9/12/2013 7:54 AM, Raheel Hasan wrote: > > I got a small issue here, my facet settings are returning counts for > empty > > "". I.e. when no the actual field was empty. > > > > Here are the facet settings: > > > > count > > 6 > > 1 > > false > > > > and this is the part of the result I dont want: > > 4 > > The "facet.missing" parameter has to do with whether or not to display > counts for documents that have no value at all for that field. > > Even though it might seem wrong, the empty string is a valid value, so > you can't fix this with faceting parameters. If you don't want that to > be in your index, then you can add the LengthFilterFactory to your > analyzer to remove terms with a length less than 1. You might also > check to see whether the field definition in your schema has a default > value set to the empty string. > > If you are using DocValues (Solr 4.2 and later), then the indexed terms > aren't used for facets, and it won't matter what you do to your analysis > chain. With DocValues, Solr basically uses a value equivalent to the > stored value. To get rid of the empty string with DocValues, you'll > need to either change your indexing process so it doesn't send empty > strings, or use a custom UpdateProcessor to change the data before it > gets indexed. > > Thanks, > Shawn > > -- Regards, Raheel Hasan
"Unable to connect" to "http://localhost:8983/solr/"
org.apache.solr.core.RequestHandlers;
created /update/extract: solr.extraction.ExtractingRequestHandler
INFO - 2013-09-12 20:08:00.525; org.apache.solr.core.RequestHandlers;
adding lazy requestHandler: solr.FieldAnalysisRequestHandler
INFO - 2013-09-12 20:08:00.525; org.apache.solr.core.RequestHandlers;
created /analysis/field: solr.FieldAnalysisRequestHandler
INFO - 2013-09-12 20:08:00.525; org.apache.solr.core.RequestHandlers;
adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
INFO - 2013-09-12 20:08:00.526; org.apache.solr.core.RequestHandlers;
created /analysis/document: solr.DocumentAnalysisRequestHandler
INFO - 2013-09-12 20:08:00.528; org.apache.solr.core.RequestHandlers;
created /admin/: solr.admin.AdminHandlers
INFO - 2013-09-12 20:08:00.530; org.apache.solr.core.RequestHandlers;
created /admin/ping: solr.PingRequestHandler
INFO - 2013-09-12 20:08:00.532; org.apache.solr.core.RequestHandlers;
created /debug/dump: solr.DumpRequestHandler
INFO - 2013-09-12 20:08:00.542; org.apache.solr.core.RequestHandlers;
created /replication: solr.ReplicationHandler
INFO - 2013-09-12 20:08:00.542; org.apache.solr.core.RequestHandlers;
adding lazy requestHandler: solr.SearchHandler
INFO - 2013-09-12 20:08:00.542; org.apache.solr.core.RequestHandlers;
created /spell: solr.SearchHandler
INFO - 2013-09-12 20:08:00.543; org.apache.solr.core.RequestHandlers;
adding lazy requestHandler: solr.SearchHandler
INFO - 2013-09-12 20:08:00.543; org.apache.solr.core.RequestHandlers;
created /tvrh: solr.SearchHandler
INFO - 2013-09-12 20:08:00.543; org.apache.solr.core.RequestHandlers;
adding lazy requestHandler: solr.SearchHandler
INFO - 2013-09-12 20:08:00.543; org.apache.solr.core.RequestHandlers;
created /terms: solr.SearchHandler
INFO - 2013-09-12 20:08:00.559; org.apache.solr.handler.loader.XMLLoader;
xsltCacheLifetimeSeconds=60
INFO - 2013-09-12 20:08:00.562; org.apache.solr.handler.loader.XMLLoader;
xsltCacheLifetimeSeconds=60
INFO - 2013-09-12 20:08:00.563; org.apache.solr.handler.loader.XMLLoader;
xsltCacheLifetimeSeconds=60
INFO - 2013-09-12 20:08:00.617; org.apache.solr.search.SolrIndexSearcher;
Opening Searcher@18e9aa3 main
INFO - 2013-09-12 20:08:00.646; org.apache.solr.update.CommitTracker; Hard
AutoCommit: if uncommited for 15000ms;
INFO - 2013-09-12 20:08:00.647; org.apache.solr.update.CommitTracker; Soft
AutoCommit: disabled
INFO - 2013-09-12 20:08:00.647;
org.apache.solr.handler.component.SpellCheckComponent; Initializing spell
checkers
INFO - 2013-09-12 20:08:00.653;
org.apache.solr.spelling.DirectSolrSpellChecker; init:
{name=default,classname=solr.DirectSolrSpellChecker,field=text,accuracy=0.5,thresholdTokenFrequency=0.01,maxEdits=1,minPrefix=3,maxInspections=3,minQueryLength=4,maxQueryFrequency=0.001}
INFO - 2013-09-12 20:08:00.660;
org.apache.solr.handler.component.SpellCheckComponent; No queryConverter
defined, using default converter
INFO - 2013-09-12 20:08:00.689;
org.apache.solr.handler.ReplicationHandler; Commits will be reserved for
1
INFO - 2013-09-12 20:08:00.690; org.apache.solr.core.QuerySenderListener;
QuerySenderListener sending requests to
Searcher@18e9aa3main{StandardDirectoryReader(segments_7i:1015
_ch(4.3):C11 _ci(4.3):C9
_cj(4.3):C1)}
INFO - 2013-09-12 20:08:00.801; org.apache.solr.request.UnInvertedField;
UnInverted multi-valued field
{field=product_code,memSize=4523,tindexSize=44,time=8,phase1=8,nTerms=2,bigTerms=2,termInstances=0,uses=0}
INFO - 2013-09-12 20:08:00.802; org.apache.solr.request.UnInvertedField;
UnInverted multi-valued field
{field=jur_code,memSize=4518,tindexSize=42,time=1,phase1=1,nTerms=2,bigTerms=2,termInstances=0,uses=0}
INFO - 2013-09-12 20:08:00.806; org.apache.solr.request.UnInvertedField;
UnInverted multi-valued field
{field=cited_id,memSize=4688,tindexSize=58,time=4,phase1=4,nTerms=231,bigTerms=0,termInstances=256,uses=0}
INFO - 2013-09-12 20:08:00.808; org.apache.solr.request.UnInvertedField;
UnInverted multi-valued field
{field=new_cited_id,memSize=4880,tindexSize=40,time=1,phase1=1,nTerms=35,bigTerms=4,termInstances=36,uses=0}
Can anyone please urgently help me?
Thanks a lot..
--
Regards,
Raheel Hasan
Re: "Unable to connect" to "http://localhost:8983/solr/"
?? anyone?
On Thu, Sep 12, 2013 at 8:12 PM, Raheel Hasan wrote:
> Hi,
>
> I just have this issue came out of no where
> Everything was fine until all of a sudden the browser cant connect to this
> solr.
>
>
> Here is the solr log:
>
> INFO - 2013-09-12 20:07:58.142; org.eclipse.jetty.server.Server;
> jetty-8.1.8.v20121106
> INFO - 2013-09-12 20:07:58.179;
> org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment monitor
> E:\Projects\G1\A1\trunk\solr_root\solrization\contexts at interval 0
> INFO - 2013-09-12 20:07:58.191;
> org.eclipse.jetty.deploy.DeploymentManager; Deployable added:
> E:\Projects\G1\A1\trunk\solr_root\solrization\contexts\solr-jetty-context.xml
> INFO - 2013-09-12 20:07:59.159;
> org.eclipse.jetty.webapp.StandardDescriptorProcessor; NO JSP Support for
> /solr, did not find org.apache.jasper.servlet.JspServlet
> INFO - 2013-09-12 20:07:59.189;
> org.eclipse.jetty.server.handler.ContextHandler; started
> o.e.j.w.WebAppContext{/solr,file:/E:/Projects/G1/A1/trunk/solr_root/solrization/solr-webapp/webapp/},E:\Projects\G1\A1\trunk\solr_root\solrization/webapps/solr.war
> INFO - 2013-09-12 20:07:59.190;
> org.eclipse.jetty.server.handler.ContextHandler; started
> o.e.j.w.WebAppContext{/solr,file:/E:/Projects/G1/A1/trunk/solr_root/solrization/solr-webapp/webapp/},E:\Projects\G1\A1\trunk\solr_root\solrization/webapps/solr.war
> INFO - 2013-09-12 20:07:59.206;
> org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
> INFO - 2013-09-12 20:07:59.231; org.apache.solr.core.SolrResourceLoader;
> JNDI not configured for solr (NoInitialContextEx)
> INFO - 2013-09-12 20:07:59.231; org.apache.solr.core.SolrResourceLoader;
> solr home defaulted to 'solr/' (could not find system property or JNDI)
> INFO - 2013-09-12 20:07:59.241;
> org.apache.solr.core.CoreContainer$Initializer; looking for solr config
> file: E:\Projects\G1\A1\trunk\solr_root\solrization\solr\solr.xml
> INFO - 2013-09-12 20:07:59.244; org.apache.solr.core.CoreContainer; New
> CoreContainer 24012447
> INFO - 2013-09-12 20:07:59.244; org.apache.solr.core.CoreContainer;
> Loading CoreContainer using Solr Home: 'solr/'
> INFO - 2013-09-12 20:07:59.245; org.apache.solr.core.SolrResourceLoader;
> new SolrResourceLoader for directory: 'solr/'
> INFO - 2013-09-12 20:07:59.483;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> socketTimeout to: 0
> INFO - 2013-09-12 20:07:59.484;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> urlScheme to: http://
> INFO - 2013-09-12 20:07:59.485;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> connTimeout to: 0
> INFO - 2013-09-12 20:07:59.486;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> maxConnectionsPerHost to: 20
> INFO - 2013-09-12 20:07:59.487;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> corePoolSize to: 0
> INFO - 2013-09-12 20:07:59.488;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> maximumPoolSize to: 2147483647
> INFO - 2013-09-12 20:07:59.489;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> maxThreadIdleTime to: 5
> INFO - 2013-09-12 20:07:59.490;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> sizeOfQueue to: -1
> INFO - 2013-09-12 20:07:59.490;
> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
> fairnessPolicy to: false
> INFO - 2013-09-12 20:07:59.498;
> org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client,
> config:maxConnectionsPerHost=20&maxConnections=1&socketTimeout=0&connTimeout=0&retry=false
> INFO - 2013-09-12 20:07:59.671; org.apache.solr.core.CoreContainer;
> Registering Log Listener
> INFO - 2013-09-12 20:07:59.689; org.apache.solr.core.CoreContainer;
> Creating SolrCore 'A1' using instanceDir: solr\A1
> INFO - 2013-09-12 20:07:59.690; org.apache.solr.core.SolrResourceLoader;
> new SolrResourceLoader for directory: 'solr\A1\'
> INFO - 2013-09-12 20:07:59.724; org.apache.solr.core.SolrConfig; Adding
> specified lib dirs to ClassLoader
> INFO - 2013-09-12 20:07:59.726; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/E:/Projects/G1/A1/trunk/solr_root/solrization/lib/mysql-connector-java-5.1.25-bin.jar'
> to classloader
> INFO - 2013-09-12 20:07:59.727; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/E:/Projects/G1/A1/trunk/solr_root/contrib/dataimporthandler/lib/activation-1.1.jar'
> to classloader
> INFO - 2013-09-12 20:07:59.727; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/E:/Projects/G1/A1/trunk/solr_ro
Re: "Unable to connect" to "http://localhost:8983/solr/"
You are right sir, its weird to have no error in the log... So after a full
day spent only on trying to figure this out, I have found the cause
(spellcheck component)... but not the solution.
Se my other post with the subject "*spellcheck causing Core Reload to hang*".
I have explained it there.
Thanks a lot.
On Fri, Sep 13, 2013 at 9:24 PM, Shawn Heisey wrote:
> On 9/13/2013 5:47 AM, Raheel Hasan wrote:
>
>> Ok I have solved it my self.. The issue was in "data" directory of
>> "solr/{myCore}/".. I deleted this folder and it started running again.
>>
>> however, this is even a bigger issue now, because when the project is LIVE
>> and it has indexed millions of records, I wont have the option to remove
>> the "data" folder again.. .
>>
>> So is there a different solution here? how to save the indexes..
>>
>
> The log you provided didn't have any error or warn messages in it, so
> there's no clue about what went wrong.
>
> If you have to delete the data directory, it usually means that your index
> is corrupt, you've changed the schema in a way that's completely
> incompatible with the existing index, or something else has gone very
> wrong. It's very weird that there's no error message in the log, though -
> problems like that typically have an error message with a long Java
> stacktrace.
>
> Thanks,
> Shawn
>
>
--
Regards,
Raheel Hasan
spellcheck causing Core Reload to hang
Hi, after a lot of investigation today, I found that its the spellcheck component which is causing the issue. If its turned off, all will run well and core can easily reload. However, when the spellcheck is on, the core wont reload instead hang forever. Then the only way to get the project back alive is to stop solr, and delete the data folder then start solr again. Here are the solr config settings for spell check: default on 5 false 5 2 false true 3 3 true spellcheck text_en_splitting default location_details solr.DirectSolrSpellChecker true 0.5 .01 1 3 3 4 0.001 Here is the field from schema: -- Regards, Raheel Hasan
what does "UnInvertedField; UnInverted multi-valued field" means and how to fix it
Hi guyz,
I have an issue here in between Solr Core and Data Indexing:
When I build some index from fresh setup, everything is fine: all queries
and additional/update indexing, everything runs is fine. But when I reload
the Core, the solr stops from that point onward forever.
All i get is this line as the last line of the solr log after the issue as
occurred:
UnInvertedField; UnInverted multi-valued field
{field=prod_cited_id,memSize=4880,tindexSize=40,time=4,phase1=4,nTerms=35,bigTerms=4,termInstances=36,uses=0}
Furthermore, the only way to get things working again, would be to delete
the "data" folder inside "solr/{myCore}/"...
So can anyone help me beat this issue and get things working again? I cant
afford this issue when the system is LIVE..
Thanks a lot.
--
Regards,
Raheel Hasan
Re: "Unable to connect" to "http://localhost:8983/solr/"
Ok I have solved it my self.. The issue was in "data" directory of
"solr/{myCore}/".. I deleted this folder and it started running again.
however, this is even a bigger issue now, because when the project is LIVE
and it has indexed millions of records, I wont have the option to remove
the "data" folder again.. .
So is there a different solution here? how to save the indexes..
On Fri, Sep 13, 2013 at 11:45 AM, Raheel Hasan wrote:
> ?? anyone?
>
>
> On Thu, Sep 12, 2013 at 8:12 PM, Raheel Hasan
> wrote:
>
>> Hi,
>>
>> I just have this issue came out of no where
>> Everything was fine until all of a sudden the browser cant connect to
>> this solr.
>>
>>
>> Here is the solr log:
>>
>> INFO - 2013-09-12 20:07:58.142; org.eclipse.jetty.server.Server;
>> jetty-8.1.8.v20121106
>> INFO - 2013-09-12 20:07:58.179;
>> org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment monitor
>> E:\Projects\G1\A1\trunk\solr_root\solrization\contexts at interval 0
>> INFO - 2013-09-12 20:07:58.191;
>> org.eclipse.jetty.deploy.DeploymentManager; Deployable added:
>> E:\Projects\G1\A1\trunk\solr_root\solrization\contexts\solr-jetty-context.xml
>> INFO - 2013-09-12 20:07:59.159;
>> org.eclipse.jetty.webapp.StandardDescriptorProcessor; NO JSP Support for
>> /solr, did not find org.apache.jasper.servlet.JspServlet
>> INFO - 2013-09-12 20:07:59.189;
>> org.eclipse.jetty.server.handler.ContextHandler; started
>> o.e.j.w.WebAppContext{/solr,file:/E:/Projects/G1/A1/trunk/solr_root/solrization/solr-webapp/webapp/},E:\Projects\G1\A1\trunk\solr_root\solrization/webapps/solr.war
>> INFO - 2013-09-12 20:07:59.190;
>> org.eclipse.jetty.server.handler.ContextHandler; started
>> o.e.j.w.WebAppContext{/solr,file:/E:/Projects/G1/A1/trunk/solr_root/solrization/solr-webapp/webapp/},E:\Projects\G1\A1\trunk\solr_root\solrization/webapps/solr.war
>> INFO - 2013-09-12 20:07:59.206;
>> org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
>> INFO - 2013-09-12 20:07:59.231; org.apache.solr.core.SolrResourceLoader;
>> JNDI not configured for solr (NoInitialContextEx)
>> INFO - 2013-09-12 20:07:59.231; org.apache.solr.core.SolrResourceLoader;
>> solr home defaulted to 'solr/' (could not find system property or JNDI)
>> INFO - 2013-09-12 20:07:59.241;
>> org.apache.solr.core.CoreContainer$Initializer; looking for solr config
>> file: E:\Projects\G1\A1\trunk\solr_root\solrization\solr\solr.xml
>> INFO - 2013-09-12 20:07:59.244; org.apache.solr.core.CoreContainer; New
>> CoreContainer 24012447
>> INFO - 2013-09-12 20:07:59.244; org.apache.solr.core.CoreContainer;
>> Loading CoreContainer using Solr Home: 'solr/'
>> INFO - 2013-09-12 20:07:59.245; org.apache.solr.core.SolrResourceLoader;
>> new SolrResourceLoader for directory: 'solr/'
>> INFO - 2013-09-12 20:07:59.483;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> socketTimeout to: 0
>> INFO - 2013-09-12 20:07:59.484;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> urlScheme to: http://
>> INFO - 2013-09-12 20:07:59.485;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> connTimeout to: 0
>> INFO - 2013-09-12 20:07:59.486;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> maxConnectionsPerHost to: 20
>> INFO - 2013-09-12 20:07:59.487;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> corePoolSize to: 0
>> INFO - 2013-09-12 20:07:59.488;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> maximumPoolSize to: 2147483647
>> INFO - 2013-09-12 20:07:59.489;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> maxThreadIdleTime to: 5
>> INFO - 2013-09-12 20:07:59.490;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> sizeOfQueue to: -1
>> INFO - 2013-09-12 20:07:59.490;
>> org.apache.solr.handler.component.HttpShardHandlerFactory; Setting
>> fairnessPolicy to: false
>> INFO - 2013-09-12 20:07:59.498;
>> org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client,
>> config:maxConnectionsPerHost=20&maxConnections=1&socketTimeout=0&connTimeout=0&retry=false
>> INFO - 2013-09-12 20:07:59.671; org.apache.solr.core.CoreContainer;
>> Registering Log Listener
>> INFO - 2013-09-12 20:07:59.689; org.apache.solr.core.CoreContainer;
>> Creating SolrCore 'A1' using instanceDir: solr\A1
>> INFO - 2013-09-12
Re: spellcheck causing Core Reload to hang
Hi, Basically, it hangs only on "core Reload" and not during queries. Furthermore, there is never any error reported in the logs, in fact the log only records until Core-Reload call. If I shut down and restart Solr, the next time it wont start, and still no errors in the log. On Sat, Sep 14, 2013 at 1:53 AM, Chris Hostetter wrote: > > : after a lot of investigation today, I found that its the spellcheck > : component which is causing the issue. If its turned off, all will run > well > : and core can easily reload. However, when the spellcheck is on, the core > : wont reload instead hang forever. > > Can you take some stack traces while the server is hung? > > Do you have any firstSearcher or newSearcher warming queries configured? > If so can you try adding "spellcheck=false" to those warming queries and > see if it eliminates the problem? > > Smells like this thread... > > https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201309.mbox/%3Calpine.DEB.2.02.1309061149310.10818@frisbee%3E > > > ...would be good to get a jira open with a reproducible set of configs > that demonstrates the problem semi-reliably.. > > > -Hoss > -- Regards, Raheel Hasan
Re: spellcheck causing Core Reload to hang
Yes I have tried Spellcheck=false and with that everything works just fine. But I do need Spell check component so I cant just leave it off. On Mon, Sep 16, 2013 at 12:24 PM, Raheel Hasan wrote: > Hi, > > Basically, it hangs only on "core Reload" and not during queries. > Furthermore, there is never any error reported in the logs, in fact the log > only records until Core-Reload call. If I shut down and restart Solr, the > next time it wont start, and still no errors in the log. > > > > > On Sat, Sep 14, 2013 at 1:53 AM, Chris Hostetter > wrote: > >> >> : after a lot of investigation today, I found that its the spellcheck >> : component which is causing the issue. If its turned off, all will run >> well >> : and core can easily reload. However, when the spellcheck is on, the core >> : wont reload instead hang forever. >> >> Can you take some stack traces while the server is hung? >> >> Do you have any firstSearcher or newSearcher warming queries configured? >> If so can you try adding "spellcheck=false" to those warming queries and >> see if it eliminates the problem? >> >> Smells like this thread... >> >> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201309.mbox/%3Calpine.DEB.2.02.1309061149310.10818@frisbee%3E >> >> >> ...would be good to get a jira open with a reproducible set of configs >> that demonstrates the problem semi-reliably.. >> >> >> -Hoss >> > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: what does "UnInvertedField; UnInverted multi-valued field" means and how to fix it
Hay, thanks for the reply.
So after a full day spent only on trying to figure this out, I have found
the cause (spellcheck component)... but not the solution.
Se my other post with the subject "*spellcheck causing Core Reload to hang*".
I have explained it there.
Thanks a lot.
On Sun, Sep 15, 2013 at 2:35 AM, Erick Erickson wrote:
> This is totally weird. Can you give us the exact
> command you are using?
>
> Best
> Erick
>
>
> On Fri, Sep 13, 2013 at 8:15 AM, Raheel Hasan >wrote:
>
> > Hi guyz,
> >
> > I have an issue here in between Solr Core and Data Indexing:
> >
> > When I build some index from fresh setup, everything is fine: all queries
> > and additional/update indexing, everything runs is fine. But when I
> reload
> > the Core, the solr stops from that point onward forever.
> >
> > All i get is this line as the last line of the solr log after the issue
> as
> > occurred:
> >
> > UnInvertedField; UnInverted multi-valued field
> >
> >
> {field=prod_cited_id,memSize=4880,tindexSize=40,time=4,phase1=4,nTerms=35,bigTerms=4,termInstances=36,uses=0}
> >
> > Furthermore, the only way to get things working again, would be to delete
> > the "data" folder inside "solr/{myCore}/"...
> >
> >
> > So can anyone help me beat this issue and get things working again? I
> cant
> > afford this issue when the system is LIVE..
> >
> > Thanks a lot.
> >
> > --
> > Regards,
> > Raheel Hasan
> >
>
--
Regards,
Raheel Hasan
Re: spellcheck causing Core Reload to hang
Please see the log (after solr restart) in the other msg I posted on this forum with the subject: "*Unable to connect" to "http://localhost:8983/solr/ *" Thanks. On Mon, Sep 16, 2013 at 12:25 PM, Raheel Hasan wrote: > Yes I have tried Spellcheck=false and with that everything works just > fine. But I do need Spell check component so I cant just leave it off. > > > On Mon, Sep 16, 2013 at 12:24 PM, Raheel Hasan > wrote: > >> Hi, >> >> Basically, it hangs only on "core Reload" and not during queries. >> Furthermore, there is never any error reported in the logs, in fact the log >> only records until Core-Reload call. If I shut down and restart Solr, the >> next time it wont start, and still no errors in the log. >> >> >> >> >> On Sat, Sep 14, 2013 at 1:53 AM, Chris Hostetter < >> hossman_luc...@fucit.org> wrote: >> >>> >>> : after a lot of investigation today, I found that its the spellcheck >>> : component which is causing the issue. If its turned off, all will run >>> well >>> : and core can easily reload. However, when the spellcheck is on, the >>> core >>> : wont reload instead hang forever. >>> >>> Can you take some stack traces while the server is hung? >>> >>> Do you have any firstSearcher or newSearcher warming queries configured? >>> If so can you try adding "spellcheck=false" to those warming queries and >>> see if it eliminates the problem? >>> >>> Smells like this thread... >>> >>> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201309.mbox/%3Calpine.DEB.2.02.1309061149310.10818@frisbee%3E >>> >>> >>> ...would be good to get a jira open with a reproducible set of configs >>> that demonstrates the problem semi-reliably.. >>> >>> >>> -Hoss >>> >> >> >> >> -- >> Regards, >> Raheel Hasan >> > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Spellcheck compounded words
Hi, I m running 4.3.. I have posted all the details in another threat... do you want me to copy it here? or could you see that? The subject is "*spellcheck causing Core Reload to hang*". On Mon, Sep 16, 2013 at 5:50 PM, Dyer, James wrote: > Which version of Solr are you running? (the post you replied to was about > Solr 3.3, but the latest version now is 4.4.) Please provide configuration > details and the query you are running that causes the problem. Also > explain exactly what the problem is (query never returns?). Also explain > why you have to delete the "data" dir when you restart. With a little > background information, maybe someone can help. > > James Dyer > Ingram Content Group > (615) 213-4311 > > -Original Message- > From: Rah1x [mailto:raheel_itst...@yahoo.com] > Sent: Monday, September 16, 2013 5:47 AM > To: solr-user@lucene.apache.org > Subject: Re: Spellcheck compounded words > > Hi guyz, > > Did anyone solve this issue? > > I am having it also, it took me 3 days to exactly figure it out that its > coming from "spellcheck.maxCollationTries"... > > Even with 1 it hangs > forewver. The only way to restart is to stop solr, delete "data" folder and > then start solr again (i.e. index lost !). > > Regards, > Raheel > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Spellcheck-compounded-words-tp3192748p4090320.html > Sent from the Solr - User mailing list archive at Nabble.com. > > > -- Regards, Raheel Hasan
Re: Spellcheck compounded words
I am building it on Commit.. true Please see my other thread for all Logs and Schema + Solrconfig settings. On Mon, Sep 16, 2013 at 7:03 PM, Dyer, James wrote: > I would investigate Hoss's suggestion and look at warming queries. In > some cases I've seen "maxCollationTries" in warming queries to cause a > hang. Unless you're trying to build your spellcheck dictionary during > warming, you can safely turn spellcheck off for all warming queries. > > James Dyer > Ingram Content Group > (615) 213-4311 > > > -Original Message- > From: Raheel Hasan [mailto:raheelhasan@gmail.com] > Sent: Monday, September 16, 2013 8:29 AM > To: solr-user@lucene.apache.org > Subject: Re: Spellcheck compounded words > > Hi, > > I m running 4.3.. > > I have posted all the details in another threat... do you want me to copy > it here? or could you see that? The subject is "*spellcheck causing Core > Reload to hang*". > > > > > On Mon, Sep 16, 2013 at 5:50 PM, Dyer, James > wrote: > > > Which version of Solr are you running? (the post you replied to was about > > Solr 3.3, but the latest version now is 4.4.) Please provide > configuration > > details and the query you are running that causes the problem. Also > > explain exactly what the problem is (query never returns?). Also explain > > why you have to delete the "data" dir when you restart. With a little > > background information, maybe someone can help. > > > > James Dyer > > Ingram Content Group > > (615) 213-4311 > > > > -Original Message- > > From: Rah1x [mailto:raheel_itst...@yahoo.com] > > Sent: Monday, September 16, 2013 5:47 AM > > To: solr-user@lucene.apache.org > > Subject: Re: Spellcheck compounded words > > > > Hi guyz, > > > > Did anyone solve this issue? > > > > I am having it also, it took me 3 days to exactly figure it out that its > > coming from "spellcheck.maxCollationTries"... > > > > Even with 1 it hangs > > forewver. The only way to restart is to stop solr, delete "data" folder > and > > then start solr again (i.e. index lost !). > > > > Regards, > > Raheel > > > > > > > > -- > > View this message in context: > > > http://lucene.472066.n3.nabble.com/Spellcheck-compounded-words-tp3192748p4090320.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > > > > > -- > Regards, > Raheel Hasan > > -- Regards, Raheel Hasan
Re: spellcheck causing Core Reload to hang
I think they should have it in RC0, because if you search in this forum at lucene, this issue is there since version 4.3 ! Regards, Raheel On Tue, Sep 17, 2013 at 5:58 PM, Erick Erickson wrote: > H, do we have a JIRA tracking this and does it seem like any fix will > get into 4.5? > > I think 4.5 RC0 will be cut tomorrow (Wednesday) > > Best, > Erick > > > On Tue, Sep 17, 2013 at 3:04 AM, Raheel Hasan >wrote: > > > I think there is another solution: > > > > Just hide this entry in solrconfig > name="spellcheck.maxCollationTries"> > > > > and instead, pass it in the actual query string that calls your > > requestHandler (like > > /select/?q=&spellcheck.maxCollationTries=3&...) > > > > > > > > On Mon, Sep 16, 2013 at 9:37 PM, Jeroen Steggink > >wrote: > > > > > Hi James, > > > > > > I already had the > > > > > > spellcheck.**collateExtendedResults=true > > > > > > Adding > > > > > > spellcheck.**collateMaxCollectDocs=0 > > > > > > did the trick. > > > > > > Thanks so much. > > > > > > Jeroen > > > > > > On 16-9-2013 18:16, Dyer, James wrote: > > > > > >> If this started with Solr4.4, I would suspect > > https://issues.apache.org/* > > >> *jira/browse/SOLR-3240 < > https://issues.apache.org/jira/browse/SOLR-3240 > > >. > > >> > > >> Rather than removing spellcheck parameters, can you try > adding/changing > > >> "spellcheck.**collateMaxCollectDocs=0" and > > "spellcheck.**collateExtendedResults=true" > > >> ? These two settings effectively disable the optimization made with > > >> SOLR-3240. > > >> > > >> James Dyer > > >> Ingram Content Group > > >> (615) 213-4311 > > >> > > >> > > > > -- > > Regards, > > Raheel Hasan > > > -- Regards, Raheel Hasan
Re: spellcheck causing Core Reload to hang
Check this thread: http://lucene.472066.n3.nabble.com/Spellcheck-compounded-words-td3192748i20.html<http://lucene.472066.n3.nabble.com/Spellcheck-compounded-words-td3192748i20.html#a4090320> This issue is there since 2011. On Tue, Sep 17, 2013 at 6:35 PM, Raheel Hasan wrote: > I think they should have it in RC0, because if you search in this forum at > lucene, this issue is there since version 4.3 ! > > Regards, > Raheel > > > On Tue, Sep 17, 2013 at 5:58 PM, Erick Erickson > wrote: > >> H, do we have a JIRA tracking this and does it seem like any fix will >> get into 4.5? >> >> I think 4.5 RC0 will be cut tomorrow (Wednesday) >> >> Best, >> Erick >> >> >> On Tue, Sep 17, 2013 at 3:04 AM, Raheel Hasan > >wrote: >> >> > I think there is another solution: >> > >> > Just hide this entry in solrconfig > > name="spellcheck.maxCollationTries"> >> > >> > and instead, pass it in the actual query string that calls your >> > requestHandler (like >> > /select/?q=&spellcheck.maxCollationTries=3&...) >> > >> > >> > >> > On Mon, Sep 16, 2013 at 9:37 PM, Jeroen Steggink > > >wrote: >> > >> > > Hi James, >> > > >> > > I already had the >> > > >> > > spellcheck.**collateExtendedResults=true >> > > >> > > Adding >> > > >> > > spellcheck.**collateMaxCollectDocs=0 >> > > >> > > did the trick. >> > > >> > > Thanks so much. >> > > >> > > Jeroen >> > > >> > > On 16-9-2013 18:16, Dyer, James wrote: >> > > >> > >> If this started with Solr4.4, I would suspect >> > https://issues.apache.org/* >> > >> *jira/browse/SOLR-3240 < >> https://issues.apache.org/jira/browse/SOLR-3240 >> > >. >> > >> >> > >> Rather than removing spellcheck parameters, can you try >> adding/changing >> > >> "spellcheck.**collateMaxCollectDocs=0" and >> > "spellcheck.**collateExtendedResults=true" >> > >> ? These two settings effectively disable the optimization made with >> > >> SOLR-3240. >> > >> >> > >> James Dyer >> > >> Ingram Content Group >> > >> (615) 213-4311 >> > >> >> > >> >> > >> > -- >> > Regards, >> > Raheel Hasan >> > >> > > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Solr SpellCheckComponent only shows results with certain fields
what about this query? try to see if you get suggestions here:
/solr/collection1/select?q=*%3Abecaus&wt=json&indent=true&spellcheck=true
On Wed, Sep 18, 2013 at 4:02 AM, jazzy wrote:
> I'm trying to get the Solr SpellCheckComponent working but am running into
> some issues. When I run
> .../solr/collection1/select?q=%3A&wt=json&indent=true
>
> These results are returned
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 1,
> "params": {
> "indent": "true",
> "q": "*:*",
> "_": "1379457032534",
> "wt": "json"
> }
> },
> "response": {
> "numFound": 2,
> "start": 0,
> "docs": [
> {
> "enterprise_name": "because",
> "name": "doc1",
> "enterprise_id": "100",
> "_version_": 1446463888248799200
> },
> {
> "enterprise_name": "what",
> "name": "RZTEST",
> "enterprise_id": "102",
> "_version_": 1446464432735518700
> }
> ]
> }
> }
> Those are the values that I have indexed. Now when I want to query for
> spelling I get some weird results.
>
> When I run
>
> .../solr/collection1/select?q=name%3Arxtest&wt=json&indent=true&spellcheck=true
>
> The results are accurate and I get
>
> {
> "responseHeader":{
> "status":0,
> "QTime":4,
> "params":{
> "spellcheck":"true",
> "indent":"true",
> "q":"name:rxtest",
> "wt":"json"}},
> "response":{"numFound":0,"start":0,"docs":[]
> },
> "spellcheck":{
> "suggestions":[
> "rxtest",{
> "numFound":1,
> "startOffset":5,
> "endOffset":11,
> "suggestion":["rztest"]}]}}
> Anytime I run a query without the name values I get 0 results back.
>
> /solr/collection1/select?q=enterprise_name%3Abecaus&wt=json&indent=true&spellcheck=true
>
> {
> "responseHeader":{
> "status":0,
> "QTime":5,
> "params":{
> "spellcheck":"true",
> "indent":"true",
> "q":"enterprise_name:becaus",
> "wt":"json"}},
> "response":{"numFound":0,"start":0,"docs":[]
> },
> "spellcheck":{
> "suggestions":[]}}
> My guess is that there is something wrong in my scheme but everything looks
> fine.
>
> Schema.xml
>
>
> required="true" />
> stored="true"/>
>
> multiValued="true" />
>
> stored="true"/>
> stored="true" multiValued="true"/>
> stored="true" multiValued="true"/>
>
>
>
>
>
> positionIncrementGap="100">
>
>
> words="stopwords.txt" />
>
>
>
>
>
> words="stopwords.txt" />
> ignoreCase="true" expand="true"/>
>
>
>
> solrconfig.xml
>
>
>
>
>explicit
>10
>text
>
>default
>
> wordbreak
>
> false
>
> false
>
> 5
>
>
>
> spellcheck
>
>
>
>
>
>
>
> default
>
> solr.IndexBasedSpellChecker
>
> name
>
> ./spellchecker
>
> 0.5
>
> .0001
> true
>
>
>
> wordbreak
> solr.WordBreakSolrSpellChecker
> name
> true
> true
> 3
> true
>
>
>
> text_general
>
>
> Any help would be appreciated.
> Thanks!
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-SpellCheckComponent-only-shows-results-with-certain-fields-tp4090727.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Regards,
Raheel Hasan
OpenJDK or OracleJDK
Hi guyz, I am trying to setup a server. Could someone tell me if OpenJDK or OracleJDK will be best for Apache Solr over CentOS? Thanks a lot. -- Regards, Raheel Hasan
Re: OpenJDK or OracleJDK
hmm why is that so? Isnt Oracle's version a bit slow? On Mon, Sep 30, 2013 at 5:56 PM, Bram Van Dam wrote: > On 09/30/2013 01:11 PM, Raheel Hasan wrote: > >> Could someone tell me if OpenJDK or OracleJDK will be best for Apache Solr >> over CentOS? >> > > If you're using Java 7 (or 8) then it doesn't matter. If you're using Java > 6, stick with the Oracle version. > > -- Regards, Raheel Hasan
Re: OpenJDK or OracleJDK
This sounds interesting... Thanks guyz for the replies.. :) On Tue, Oct 1, 2013 at 8:07 AM, Otis Gospodnetic wrote: > Hi, > > A while back I remember we notices some SPM users were having issues > with OpenJDK. Since then we've been recommending Oracle's > implementation to our Solr and to SPM users. At the same time, we > haven't seen any issues with OpenJDK in the last ~6 months. Oracle > JDK is not slow. :) > > Otis > -- > Solr & ElasticSearch Support -- http://sematext.com/ > Performance Monitoring -- http://sematext.com/spm > > > > On Mon, Sep 30, 2013 at 11:02 PM, Shawn Heisey wrote: > > On 9/30/2013 9:28 AM, Raheel Hasan wrote: > >> hmm why is that so? > >> Isnt Oracle's version a bit slow? > > > > For Java 6, the Sun JDK is the reference implementation. For Java 7, > > OpenJDK is the reference implementation. > > > > http://en.wikipedia.org/wiki/Reference_implementation > > > > I don't think Oracle's version could really be called slow. Sun > > invented Java. Sun open sourced Java. Oracle bought Sun. > > > > The Oracle implemetation is likely more conservative than some of the > > other implementations, like the one by IBM. The IBM implementation is > > pretty aggressive with optimization, so aggressive that Solr and Lucene > > have a history of revealing bugs that only exist in that implementation. > > > > Thanks, > > Shawn > > > -- Regards, Raheel Hasan
Spell Checker (DirectSolrSpellChecker) correct settings
Hi guyz, I am new to solr. Here is the thing I have:
When i search "Courtt", I get correct suggestion saying:
"spellcheck": {
"suggestions": [
"courtt",
{
"numFound": 1,
"startOffset": 0,
"endOffset": 6,
"suggestion": [
"court"
]
},
"collation",
[
"collationQuery",
"court",
"hits",
53,
"misspellingsAndCorrections",
[
"courtt",
"court"
]
]
]
},
But when I try "Provincial Courtt", it gives me no suggestions, instead it
searches for "Provincial" only.
Here is the spell check settings in *solrconfig.xml*:
text_en_splitting
default
solr.DirectSolrSpellChecker
text
0.5
.01
1
3
3
4
0.01
wordbreak
solr.WordBreakSolrSpellChecker
text
true
true
5
Here is the *requestHandler*:
explicit
20
text
on
5
true
5
2
false
true
3
3
true
spellcheck
--
Regards,
Raheel Hasan
Re: Spell Checker (DirectSolrSpellChecker) correct settings
anyone?
On Fri, May 31, 2013 at 7:37 PM, Raheel Hasan wrote:
> Hi guyz, I am new to solr. Here is the thing I have:
>
> When i search "Courtt", I get correct suggestion saying:
>
>
>
> "spellcheck": {
> "suggestions": [
> "courtt",
> {
> "numFound": 1,
> "startOffset": 0,
> "endOffset": 6,
> "suggestion": [
> "court"
> ]
> },
> "collation",
> [
> "collationQuery",
> "court",
> "hits",
> 53,
> "misspellingsAndCorrections",
> [
> "courtt",
> "court"
> ]
> ]
> ]
> },
>
>
>
> But when I try "Provincial Courtt", it gives me no suggestions, instead it
> searches for "Provincial" only.
>
>
> Here is the spell check settings in *solrconfig.xml*:
>
>
> text_en_splitting
>
>
>
> default
> solr.DirectSolrSpellChecker
> text
>
>
> 0.5
>
> .01
>
>
>
> 1
>
> 3
>
> 3
>
> 4
>
> 0.01
>
>
>
>
>
> wordbreak
> solr.WordBreakSolrSpellChecker
> text
> true
> true
> 5
>
>
>
>
>
> Here is the *requestHandler*:
>
>
>
>explicit
>20
>text
>
>
>on
>5
>true
>5
>2
>false
>
>true
>3
>3
>true
>
>
>
>
>spellcheck
>
>
>
>
>
>
> --
> Regards,
> Raheel Hasan
>
--
Regards,
Raheel Hasan
/non/existent/dir/yields/warning
Hi, I am constantly getting this error in my solr log: Can't find (or read) directory to add to classloader: /non/existent/dir/yields/warning (resolved as: E:\Projects\apache_solr\solr-4.3.0\example\solr\genesis_experimental\non\existent\dir\yields\warning). Anyone got any idea on how to solve this -- Regards, Raheel Hasan
Re: /non/existent/dir/yields/warning
ok thanks :) But why was it there anyway? I mean it says in comments: If a 'dir' option (with or without a regex) is used and nothing is found that matches, a warning will be logged. So it looks like a kind of exception handling or logging for libs not found... so shouldnt this folder actually exist? On Mon, Jun 3, 2013 at 2:06 PM, Rafał Kuć wrote: > Hello! > > You should remove that entry from your solrconfig.xml file. It is > something like this: > > > > > -- > Regards, > Rafał Kuć > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > > > Hi, > > > I am constantly getting this error in my solr log: > > > Can't find (or read) directory to add to classloader: > > /non/existent/dir/yields/warning (resolved as: > > > E:\Projects\apache_solr\solr-4.3.0\example\solr\genesis_experimental\non\existent\dir\yields\warning). > > > Anyone got any idea on how to solve this > > > -- Regards, Raheel Hasan
Re: Spell Checker (DirectSolrSpellChecker) correct settings
Let me repeat the issue in brief:
When i search "*Courtt*", I get correct suggestion. But when I try "*Provincial
Courtt*", it gives me no suggestions, instead it searches for "Provincial"
only.
All the config done in my test project is in the original email.
Any help would be great...
Thanks a lot.
On Mon, Jun 3, 2013 at 1:33 PM, Raheel Hasan wrote:
> anyone?
>
>
> On Fri, May 31, 2013 at 7:37 PM, Raheel Hasan
> wrote:
>
>> Hi guyz, I am new to solr. Here is the thing I have:
>>
>> When i search "Courtt", I get correct suggestion saying:
>>
>>
>>
>> "spellcheck": {
>> "suggestions": [
>> "courtt",
>> {
>> "numFound": 1,
>> "startOffset": 0,
>> "endOffset": 6,
>> "suggestion": [
>> "court"
>> ]
>> },
>> "collation",
>> [
>> "collationQuery",
>> "court",
>> "hits",
>> 53,
>> "misspellingsAndCorrections",
>> [
>> "courtt",
>> "court"
>> ]
>> ]
>> ]
>> },
>>
>>
>>
>> But when I try "Provincial Courtt", it gives me no suggestions, instead
>> it searches for "Provincial" only.
>>
>>
>> Here is the spell check settings in *solrconfig.xml*:
>>
>>
>> text_en_splitting
>>
>>
>>
>> default
>> solr.DirectSolrSpellChecker
>> text
>>
>>
>> 0.5
>>
>> .01
>>
>>
>>
>> 1
>>
>> 3
>>
>> 3
>>
>> 4
>>
>> 0.01
>>
>>
>>
>>
>>
>> wordbreak
>> solr.WordBreakSolrSpellChecker
>> text
>> true
>> true
>> 5
>>
>>
>>
>>
>>
>> Here is the *requestHandler*:
>>
>>
>>
>>explicit
>>20
>>text
>>
>>
>>on
>>5
>>true
>>5
>>2
>>false
>>
>>true
>>3
>>3
>>true
>>
>>
>>
>>
>>spellcheck
>>
>>
>>
>>
>>
>>
>> --
>> Regards,
>> Raheel Hasan
>>
>
>
>
> --
> Regards,
> Raheel Hasan
>
--
Regards,
Raheel Hasan
Re: /non/existent/dir/yields/warning
Hi, but the path looks like it shows how to setup non existent lib warning... :D On Mon, Jun 3, 2013 at 2:56 PM, Rafał Kuć wrote: > Hello! > > That's a good question. I suppose its there to show users how to setup > a custom path to libraries. > > -- > Regards, > Rafał Kuć > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > > > ok thanks :) > > > But why was it there anyway? I mean it says in comments: > > If a 'dir' option (with or without a regex) is used and nothing > > is found that matches, a warning will be logged. > > > So it looks like a kind of exception handling or logging for libs not > > found... so shouldnt this folder actually exist? > > > > > > > On Mon, Jun 3, 2013 at 2:06 PM, Rafał Kuć wrote: > > >> Hello! > >> > >> You should remove that entry from your solrconfig.xml file. It is > >> something like this: > >> > >> > >> > >> > >> -- > >> Regards, > >> Rafał Kuć > >> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - > ElasticSearch > >> > >> > Hi, > >> > >> > I am constantly getting this error in my solr log: > >> > >> > Can't find (or read) directory to add to classloader: > >> > /non/existent/dir/yields/warning (resolved as: > >> > > >> > E:\Projects\apache_solr\solr-4.3.0\example\solr\genesis_experimental\non\existent\dir\yields\warning). > >> > >> > Anyone got any idea on how to solve this > >> > >> > >> > > > -- Regards, Raheel Hasan
Multitable import - uniqueKey
Hi, I am importing multiple table (by join) into solr using DIH. All is set, except for 1 confusion: what to do with "**" in schema? When I had only 1 table, I had it fine. Now how to put 2 uniqueKeys (both from different table). For example: table1_id table2_id Will this work? -- Regards, Raheel Hasan
Re: Multitable import - uniqueKey
Hi, Thanks for the replies. Actually, I had only a small confusion: >From table_1 I got key_1; using this I join into table_2. But table_2 also gave another key key_2 which is needed for joining with table_3. So for Table1 and Table2 its obviously just fine... but what will happen when table3 is also added? will the 3 tables be intact in terms of relationship? Thanks. On Mon, Jun 3, 2013 at 7:33 PM, Jack Krupansky wrote: > If the respective table IDs are not globally unique, then you (the > developer) will have to supplement the raw ID with a prefix or suffix or > other form of global ID (e.g., UUID) to assure that they are unique. You > could just add the SQL table name as a prefix or suffix. > > The bottom line: What do you WANT the Solr key field to look like? I mean, > YOU are the data architect, right? What requirements do you have? When your > Solr application users receive the key values in the responses to queries, > what expectations do you expect to set for them? > > -- Jack Krupansky > > -----Original Message- From: Raheel Hasan > Sent: Monday, June 03, 2013 9:12 AM > To: solr-user@lucene.apache.org > Subject: Multitable import - uniqueKey > > > Hi, > > I am importing multiple table (by join) into solr using DIH. All is set, > except for 1 confusion: > what to do with "**" in schema? > > > When I had only 1 table, I had it fine. Now how to put 2 uniqueKeys (both > from different table). > > For example: > > table1_id > table2_id > > Will this work? > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: /non/existent/dir/yields/warning
ok fantastic... now I will comment it to be sure thanks a lot Regards, Raheel On Mon, Jun 3, 2013 at 7:27 PM, Shawn Heisey wrote: > On 6/3/2013 5:58 AM, Raheel Hasan wrote: > > but the path looks like it shows how to setup non existent lib warning... > > :D > > The reason for its existence is encoded in its name. A nonexistent path > results in a warning. It's a way to illustrate to a novice what happens > when you have a non-fatal misconfiguration. The message is a warning > and doesn't prevent Solr startup. > > Thanks, > Shawn > > -- Regards, Raheel Hasan
Re: Multitable import - uniqueKey
ok. But do we need it? Thats what I am confused at. should 1 key from table_1 pull all the data in relationship as they were inserted? On Mon, Jun 3, 2013 at 7:53 PM, Jack Krupansky wrote: > Same answer. Whether it is 2, 3, 10 or 1000 tables, you, the data > architect must decide how to uniquely identify Solr documents. In general, > when joining n tables, combine the n keys into one composite key. Either do > it on the SQL query side, or with a Solr update request processor. > > > -- Jack Krupansky > > -Original Message- From: Raheel Hasan > Sent: Monday, June 03, 2013 10:44 AM > To: solr-user@lucene.apache.org > Subject: Re: Multitable import - uniqueKey > > > Hi, > > Thanks for the replies. Actually, I had only a small confusion: > > From table_1 I got key_1; using this I join into table_2. But table_2 also > gave another key key_2 which is needed for joining with table_3. > > So for Table1 and Table2 its obviously just fine... but what will happen > when table3 is also added? will the 3 tables be intact in terms of > relationship? > > Thanks. > > > > On Mon, Jun 3, 2013 at 7:33 PM, Jack Krupansky ** > wrote: > > If the respective table IDs are not globally unique, then you (the >> developer) will have to supplement the raw ID with a prefix or suffix or >> other form of global ID (e.g., UUID) to assure that they are unique. You >> could just add the SQL table name as a prefix or suffix. >> >> The bottom line: What do you WANT the Solr key field to look like? I mean, >> YOU are the data architect, right? What requirements do you have? When >> your >> Solr application users receive the key values in the responses to queries, >> what expectations do you expect to set for them? >> >> -- Jack Krupansky >> >> -Original Message- From: Raheel Hasan >> Sent: Monday, June 03, 2013 9:12 AM >> To: solr-user@lucene.apache.org >> Subject: Multitable import - uniqueKey >> >> >> Hi, >> >> I am importing multiple table (by join) into solr using DIH. All is set, >> except for 1 confusion: >> what to do with "**" in schema? >> >> >> When I had only 1 table, I had it fine. Now how to put 2 uniqueKeys (both >> from different table). >> >> For example: >> >> table1_id >> table2_id >> >> >> Will this work? >> >> -- >> Regards, >> Raheel Hasan >> >> > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Re: Spell Checker (DirectSolrSpellChecker) correct settings
Hi, thanks a lot for the reply.
Actually, "Provincial Courtt" is mentioned in many documents (sorry about
the type earlier).
Secondly, I tried your idea, but not much of help. The issue is very
microscopic:
1) When I search for "Provinciaal Courtt" = it only suggests `court` and not "Provincial"
2) Search for "Provincial Courtt" = returns result for 'Provincial' keyword
and no suggestion for 'court'.
3) Search for "Provinciaal Court" = no suggestion; instead searches for
"court" and returns result.
4) Search for "Provinciall Courtt" = correct suggestions..
On Mon, Jun 3, 2013 at 7:55 PM, Dyer, James wrote:
> My first guess is that no documents match the query "provinical court".
> Because you have "spellcheck.maxCollationTries" set to a non-zero value,
> it will not return these as collations unless the correction will return
> hits. You can test my theory out by removing
> "spellcheck.maxCollationTries" from the request and see if it returns
> "provinical court" as expected.
>
> If this isn't it, then give us the full query request and also the full
> spellcheck response for your failing case.
>
> James Dyer
> Ingram Content Group
> (615) 213-4311
>
>
> -Original Message-
> From: Raheel Hasan [mailto:raheelhasan@gmail.com]
> Sent: Friday, May 31, 2013 9:38 AM
> To: solr-user@lucene.apache.org
> Subject: Spell Checker (DirectSolrSpellChecker) correct settings
>
> Hi guyz, I am new to solr. Here is the thing I have:
>
> When i search "Courtt", I get correct suggestion saying:
>
>
>
> "spellcheck": {
> "suggestions": [
> "courtt",
> {
> "numFound": 1,
> "startOffset": 0,
> "endOffset": 6,
> "suggestion": [
> "court"
> ]
> },
> "collation",
> [
> "collationQuery",
> "court",
> "hits",
> 53,
> "misspellingsAndCorrections",
> [
> "courtt",
> "court"
> ]
> ]
> ]
> },
>
>
>
> But when I try "Provincial Courtt", it gives me no suggestions, instead it
> searches for "Provincial" only.
>
>
> Here is the spell check settings in *solrconfig.xml*:
>
>
> text_en_splitting
>
>
>
> default
> solr.DirectSolrSpellChecker
> text
>
>
> 0.5
>
> .01
>
>
>
> 1
>
> 3
>
> 3
>
> 4
>
> 0.01
>
>
>
>
>
> wordbreak
> solr.WordBreakSolrSpellChecker
> text
> true
> true
> 5
>
>
>
>
>
> Here is the *requestHandler*:
>
>
>
>explicit
>20
>text
>
>
>on
>5
>true
>5
>2
>false
>
>true
>3
>3
>true
>
>
>
>
>spellcheck
>
>
>
>
>
>
> --
> Regards,
> Raheel Hasan
>
>
--
Regards,
Raheel Hasan
Re: Multitable import - uniqueKey
oh thanks a lot for the replies. Table 1, 2, 3 are just "normalized" form of the whole record and you can easily pull one from another... but thats RDBMS, I dont know if thats true in Solr as well. On Mon, Jun 3, 2013 at 8:33 PM, Shawn Heisey wrote: > > ok. But do we need it? Thats what I am confused at. should 1 key from > > table_1 pull all the data in relationship as they were inserted? > > Suoplementing all the great info Jack has relayed: > > For the unique key in schema.xml, you really only need to be concerned > about the document uniqueness within Solr, not the database. > > Are the entries from table 2 (and table 3, etc) full siblings to the > entries in table 1? If they are, and you can't guarantee they will have > different ID values from table 1, then you need some kind of composite > key. > > If tables 2 and 3 are used to populate data in documents from table 1, > then you don't need to worry about their unique keys in Solr. Solr already > has enough information to uniquely id any of the top level docs. > > Thanks, > Shawn > > > -- Regards, Raheel Hasan
Solr directories in 4.3
Hi, I am starting a fresh project. Could someone please explain to me what folders are needed here? [image: Inline image 1] I know "solr" and "lib" are needed. But what are all these other folders there for? Thanks. -- Regards, Raheel Hasan
Re: Solr directories in 4.3
ok, here you go: http://picpaste.com/solr_folders-JYwLHoRP.png On Tue, Jun 4, 2013 at 2:45 PM, Yago Riveiro wrote: > Raheel, > > I can see the image, the mailing list doesn't work well with images, maybe > you can upload the image in a site like http://picpaste.com/ and after > share the link. > > -- > Yago Riveiro > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Tuesday, June 4, 2013 at 10:39 AM, Raheel Hasan wrote: > > > Hi, > > > > I am starting a fresh project. Could someone please explain to me what > folders are needed here? > > > > I know "solr" and "lib" are needed. But what are all these other folders > there for? > > > > Thanks. > > > > -- > > Regards, > > Raheel Hasan > > -- Regards, Raheel Hasan
Re: Solr directories in 4.3
ok thank alot for the fantastic explanation. Maybe your reply should be added into the README file that is in that folder.. On Tue, Jun 4, 2013 at 3:47 PM, Daniel Collins wrote: > The example is just that, its an example, not a cast iron base to work > from. > > contexts, etc, lib, resources, solr-webapp and webapps are part of/related > to the Jetty deployment. You might not need all the files in them though. > > cloud-scripts is just some sample scripts for accessing ZK (optional and > only if you are using Solr cloud). > > example-DIH, examplesdocs are various example things (as the names > suggest), either data or Data Import Handler related configuration > > multicore is an equivalent of solr for multiple cores, so you probably > don't need that. > > solr contains the configuration, schema, and solr.xml/zoo.cfg (which I > believe are being deprecated in favor of configuration in ZK itself). > > Certainly, you can remove the 2 example directories, the rest you'll have > to look through and work out if you need it. It depends exactly what > configuration you are trying to create as to what you need (single > standalone core, multiple cores, Solr Cloud, etc) but that should give you > a start. > > > > On 4 June 2013 11:01, Yago Riveiro wrote: > > > Raheel, > > > > The folders are part of the jetty embedded into solr for run it. > > > > -- > > Yago Riveiro > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > On Tuesday, June 4, 2013 at 10:49 AM, Raheel Hasan wrote: > > > > > ok, here you go: > > > http://picpaste.com/solr_folders-JYwLHoRP.png > > > > > > > > > On Tue, Jun 4, 2013 at 2:45 PM, Yago Riveiro (mailto: > > yago.rive...@gmail.com)> wrote: > > > > > > > Raheel, > > > > > > > > I can see the image, the mailing list doesn't work well with images, > > maybe > > > > you can upload the image in a site like http://picpaste.com/ and > after > > > > share the link. > > > > > > > > -- > > > > Yago Riveiro > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > > > > > On Tuesday, June 4, 2013 at 10:39 AM, Raheel Hasan wrote: > > > > > > > > > Hi, > > > > > > > > > > I am starting a fresh project. Could someone please explain to me > > what > > > > folders are needed here? > > > > > > > > > > I know "solr" and "lib" are needed. But what are all these other > > folders > > > > there for? > > > > > > > > > > Thanks. > > > > > > > > > > -- > > > > > Regards, > > > > > Raheel Hasan > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Regards, > > > Raheel Hasan > > > > > > > > > > > > > -- Regards, Raheel Hasan
Re: Spell Checker (DirectSolrSpellChecker) correct settings
Here is the query request string:
http://localhost:8983/solr/.../select?q=Provinci+Courtt&wt=xml&indent=true
So you see, its just simple. Thats because all items are defined in the
solrconfig.xml requestHandler for select, instead of being passed in the
query string. They were all posted in the first msg of this conversation.
»
Here is the idea:
You google up the following: "ok thank alot for the fantastic explaination"
and you see the suggestions: "Did you mean: ok *thanks *alot for the
fantastic *explanation*"
I cant build this without getting corect suggestions...
«--
On Mon, Jun 3, 2013 at 8:44 PM, Dyer, James wrote:
> For each fot he 4 cases listed below, can you give your query request
> string (q=...fq=...qt=...etc) and also the spellchecker output?
>
> James Dyer
> Ingram Content Group
> (615) 213-4311
>
>
> -Original Message-
> From: Raheel Hasan [mailto:raheelhasan@gmail.com]
> Sent: Monday, June 03, 2013 10:22 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Spell Checker (DirectSolrSpellChecker) correct settings
>
> Hi, thanks a lot for the reply.
>
> Actually, "Provincial Courtt" is mentioned in many documents (sorry about
> the type earlier).
>
> Secondly, I tried your idea, but not much of help. The issue is very
> microscopic:
>
> 1) When I search for "Provinciaal Courtt" = it only suggests ` "courtt">court` and not "Provincial"
> 2) Search for "Provincial Courtt" = returns result for 'Provincial' keyword
> and no suggestion for 'court'.
> 3) Search for "Provinciaal Court" = no suggestion; instead searches for
> "court" and returns result.
> 4) Search for "Provinciall Courtt" = correct suggestions..
>
>
>
>
>
>
> On Mon, Jun 3, 2013 at 7:55 PM, Dyer, James >wrote:
>
> > My first guess is that no documents match the query "provinical court".
> > Because you have "spellcheck.maxCollationTries" set to a non-zero value,
> > it will not return these as collations unless the correction will return
> > hits. You can test my theory out by removing
> > "spellcheck.maxCollationTries" from the request and see if it returns
> > "provinical court" as expected.
> >
> > If this isn't it, then give us the full query request and also the full
> > spellcheck response for your failing case.
> >
> > James Dyer
> > Ingram Content Group
> > (615) 213-4311
> >
> >
> > -Original Message-
> > From: Raheel Hasan [mailto:raheelhasan@gmail.com]
> > Sent: Friday, May 31, 2013 9:38 AM
> > To: solr-user@lucene.apache.org
> > Subject: Spell Checker (DirectSolrSpellChecker) correct settings
> >
> > Hi guyz, I am new to solr. Here is the thing I have:
> >
> > When i search "Courtt", I get correct suggestion saying:
> >
> >
> >
> > "spellcheck": {
> > "suggestions": [
> > "courtt",
> > {
> > "numFound": 1,
> > "startOffset": 0,
> > "endOffset": 6,
> > "suggestion": [
> > "court"
> > ]
> > },
> > "collation",
> > [
> > "collationQuery",
> > "court",
> > "hits",
> > 53,
> > "misspellingsAndCorrections",
> > [
> > "courtt",
> > "court"
> > ]
> > ]
> > ]
> > },
> >
> >
> >
> > But when I try "Provincial Courtt", it gives me no suggestions, instead
> it
> > searches for "Provincial" only.
> >
> >
> > Here is the spell check settings in *solrconfig.xml*:
> >
> >
> > text_en_splitting
> >
> >
> >
> > default
> > solr.DirectSolrSpellChecker
> > text
> >
> >
> > 0.5
> >
> > .01
> >
> >
> >
> > 1
> >
> > 3
> >
> > 3
> >
> > 4
> >
> > 0.01
> >
> >
> >
> >
> >
> > wordbreak
> > solr.WordBreakSolrSpellChecker
> > text
> > true
> > true
> > 5
> >
> >
> >
> >
> >
> > Here is the *requestHandler*:
> >
> >
> >
> >explicit
> >20
> >text
> >
> >
> >on
> >5
> >true
> >5
> >2
> >false
> >
> >true
> >3
> >3
> >true
> >
> >
> >
> >
> >spellcheck
> >
> >
> >
> >
> >
> >
> > --
> > Regards,
> > Raheel Hasan
> >
> >
>
>
> --
> Regards,
> Raheel Hasan
>
>
--
Regards,
Raheel Hasan
Re: Why is my DIH delta import doing a full import?
maybe this will help you:
http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
On Tue, Jun 4, 2013 at 8:38 PM, Arun Rangarajan wrote:
> Shawn,
>
> Thanks for your reply. My data-config.xml actually has two entities. I sent
> only the first entity in my previous email. Since I had not run any imports
> on the 2nd entity, dataimport.properties did not have an entry for it yet.
> This worked fine in 3.6.2, so looks like a bug in 4.2.1.
>
> For now, I am thinking that I can skip using the dih properties entirely.
> For the first entity, I can look for documents that changed in the last 10
> min in the DB and run the delta import cron job every 10 min. For the 2nd
> entity, the interval is 1 hour. Of course, if one of the delta imports fail
> this approach may skip some documents, but we do full import once a day so
> those docs should eventually catch up. Guess that's the best I can get with
> DIH for now!
>
>
> On Tue, Jun 4, 2013 at 7:05 AM, Shawn Heisey wrote:
>
> > On 6/4/2013 7:52 AM, Arun Rangarajan wrote:
> > > I upgraded from Solr 3.6.2 to 4.2.1 and I am noticing that my data
> import
> > > handler's delta import is actually doing a full import.
> >
> >
> >
> > > What changed and how do I get delta import to only index the documents
> > that
> > > got modified after ${dih.Lists.last_index_time}'?
> >
> > It's a bug. I've built a test that shows the problem, but I haven't
> > figured out yet how to actually fix it.
> >
> > https://issues.apache.org/jira/browse/SOLR-4788
> >
> > I now have one more data point to add to the mix that I didn't know
> > before - it works in 3.6.2.
> >
> > It looks like you only have the one entity showing a last_indexed_time,
> > so you should be able to use ${dih.last_index_time} instead of
> > ${dih.Lists.last_index_time}.
> >
> > Thanks,
> > Shawn
> >
> >
>
--
Regards,
Raheel Hasan
Indexing Heavy dataset
Hi, I am trying to index a heavy dataset with 1 particular field really too heavy... However, As I start, I get Memory warning and rollback (OutOfMemoryError). So, I have learned that we can use -Xmx1024m option with java command to start the solr and allocate more memory to the heap. My question is, that since this could also become insufficient later, so it the issue related to cacheing? here is my cache block in solrconfig: I am thinking like maybe I need to turn of the cache for "documentClass". Anyone got a better idea? Or perhaps there is another issue here? Just to let you know, until I added that very heavy db field for indexing, everything was just fine... -- Regards, Raheel Hasan
Re: Heap space problem with mlt query
and I just asked a similar question just 1 sec ago On Wed, Jun 5, 2013 at 2:07 PM, Varsha Rani wrote: > Hi , > > I am having solr index of 80GB with 1 million documents .Each document of > aprx. 500KB . I have a machine with 16GB ram. > > I am running mlt query on 3-5 fields of theses document . > > I am getting solr out of memory problem . > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > > My Solr config is : > > 128 > 100 > 1 > 1000 > 1 > > > I also checked with ramBuffer size of 256MB. > > Please provide me suggestion regarding this. > > Thanks > Varsha > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Heap-space-problem-with-mlt-query-tp4068278.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Regards, Raheel Hasan
different Solr Logging for CONSOLE and FILE
Hi, I have a small question about solr logging. In resources>log4j.properties, we have *log4j.rootLogger=INFO, file, CONSOLE* However, what I want is: *log4j.rootLogger=INFO, file * and *log4j.rootLogger=WARN, CONSOLE* (both simultaneously). Is it possible? -- Regards, Raheel Hasan
Re: different Solr Logging for CONSOLE and FILE
OK thanks... it works... :D Also I found that we could put both of them and it will also work: log4j.rootLogger=INFO, file log4j.rootLogger=WARN, CONSOLE On Wed, Jun 5, 2013 at 2:42 PM, Bernd Fehling < bernd.fehl...@uni-bielefeld.de> wrote: > > > Am 05.06.2013 11:28, schrieb Raheel Hasan: > > Hi, > > > > I have a small question about solr logging. > > > > In resources>log4j.properties, we have > > > > *log4j.rootLogger=INFO, file, CONSOLE* > > > > However, what I want is: > > *log4j.rootLogger=INFO, file > > * > > and > > *log4j.rootLogger=WARN, CONSOLE* > > (both simultaneously). > > > > Is it possible? > > > > You can use: > > log4j.rootLogger=INFO, file, CONSOLE > > log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender > log4j.appender.CONSOLE.Threshold=WARN > > -- Regards, Raheel Hasan
Files included from the default SolrConfig
Hi, I am trying to optimize solr. The default solrConfig that comes with solr>collection1 has a lot of libs included I dont really need. Perhaps if someone could help we identifying the purpose. (I only import from DIH): Please tell me whats in these: contrib/extraction/lib solr-cell- contrib/clustering/lib solr-clustering- contrib/langid/lib/ solr-langid -- Regards, Raheel Hasan
Re: Indexing Heavy dataset
ok thanks for the reply The field having values like 60kb each Furthermore, I have realized that the issue is with MySQL as its not processing this table when a "where" is applied Secondly, I have turned this field to "*stored=false*" and now the "*select/ *" is fast working again On Wed, Jun 5, 2013 at 6:56 PM, Shawn Heisey wrote: > On 6/5/2013 3:08 AM, Raheel Hasan wrote: > > Hi, > > > > I am trying to index a heavy dataset with 1 particular field really too > > heavy... > > > > However, As I start, I get Memory warning and rollback > (OutOfMemoryError). > > So, I have learned that we can use -Xmx1024m option with java command to > > start the solr and allocate more memory to the heap. > > > > My question is, that since this could also become insufficient later, so > it > > the issue related to cacheing? > > > > here is my cache block in solrconfig: > > > > > size="512" > > initialSize="512" > > autowarmCount="0"/> > > > > > size="512" > > initialSize="512" > > autowarmCount="0"/> > > > > >size="512" > >initialSize="512" > >autowarmCount="0"/> > > > > I am thinking like maybe I need to turn of the cache for "documentClass". > > Anyone got a better idea? Or perhaps there is another issue here? > > Exactly how big is this field? Do you need this giant field returned > with your results, or is it just there for searching? > > Caches of size 512, especially with autowarm disabled, are probably not > a major cause for concern, unless the big field is big enough so that > 512 of them is really really huge. If that's the case, I would reduce > the size of your documentCache, not turn it off. > > The value of ramBufferSizeMB elsewhere in your config is more likely to > affect how much RAM gets used during indexing. The default for this > field as of Solr 4.1.0 is 100. Most people can reduce this value. > > I'm writing a reply to another thread where you are participating, with > info that will likely be useful for you too. Look for that. > > Thanks, > Shawn > > -- Regards, Raheel Hasan
Re: Indexing Heavy dataset
some values in the field are up to a 1M as well On Wed, Jun 5, 2013 at 7:27 PM, Raheel Hasan wrote: > ok thanks for the reply The field having values like 60kb each > > Furthermore, I have realized that the issue is with MySQL as its not > processing this table when a "where" is applied > > Secondly, I have turned this field to "*stored=false*" and now the "* > select/*" is fast working again > > > > On Wed, Jun 5, 2013 at 6:56 PM, Shawn Heisey wrote: > >> On 6/5/2013 3:08 AM, Raheel Hasan wrote: >> > Hi, >> > >> > I am trying to index a heavy dataset with 1 particular field really too >> > heavy... >> > >> > However, As I start, I get Memory warning and rollback >> (OutOfMemoryError). >> > So, I have learned that we can use -Xmx1024m option with java command to >> > start the solr and allocate more memory to the heap. >> > >> > My question is, that since this could also become insufficient later, >> so it >> > the issue related to cacheing? >> > >> > here is my cache block in solrconfig: >> > >> > > > size="512" >> > initialSize="512" >> > autowarmCount="0"/> >> > >> > > > size="512" >> > initialSize="512" >> > autowarmCount="0"/> >> > >> > > >size="512" >> >initialSize="512" >> >autowarmCount="0"/> >> > >> > I am thinking like maybe I need to turn of the cache for >> "documentClass". >> > Anyone got a better idea? Or perhaps there is another issue here? >> >> Exactly how big is this field? Do you need this giant field returned >> with your results, or is it just there for searching? >> >> Caches of size 512, especially with autowarm disabled, are probably not >> a major cause for concern, unless the big field is big enough so that >> 512 of them is really really huge. If that's the case, I would reduce >> the size of your documentCache, not turn it off. >> >> The value of ramBufferSizeMB elsewhere in your config is more likely to >> affect how much RAM gets used during indexing. The default for this >> field as of Solr 4.1.0 is 100. Most people can reduce this value. >> >> I'm writing a reply to another thread where you are participating, with >> info that will likely be useful for you too. Look for that. >> >> Thanks, >> Shawn >> >> > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
Custom Data Clustering
Hi, Can someone please tell me if there is a way to have a custom *`clustering of the data`* from `solr` 'query' results? I am facing 2 issues currently: 1. The `*Carrot*` clustering only applies clustering to the "paged" results (i.e. in the current pagination's page results). 2. I need to have custom clustering and classify results into certain classes only (i.e. only few very specific words in the search results). Like for example "Red", "Green", "Blue" etc... and not "hello World", "Known World", "green world" etc -(if you know what I mean here) - Where all these words in both Do and DoNot existing in the search results. Please tell me how to achieve this. Perhaps Carrot/clustering is not needed here and some other classifier is needed. So what to do here? Basically, I cannot receive 1 million results, then process them via PHP-Array to classify them as per need. The classification must be done here in solr only. Thanks -- Regards, Raheel Hasan
Re: Custom Data Clustering
I wounder how to do that shouldn't this already be part of Solr? Also, I read over then the Internet that it possible to use Mahout and Solr for this purpose so how to achieve that? On Sun, Jun 9, 2013 at 7:57 AM, Otis Gospodnetic wrote: > Hello, > > This sounds like a custom SearchComponent. > Which clustering library you want to use or DIY is up to you, but go > with the SearchComponent approach. You will still need to process N > hits, but you won't need to first send them all over the wire. > > Otis > -- > Solr & ElasticSearch Support > http://sematext.com/ > > > > > > On Fri, Jun 7, 2013 at 11:48 AM, Raheel Hasan > wrote: > > Hi, > > > > Can someone please tell me if there is a way to have a custom > *`clustering > > of the data`* from `solr` 'query' results? I am facing 2 issues > currently: > > > > 1. The `*Carrot*` clustering only applies clustering to the "paged" > > results (i.e. in the current pagination's page results). > > > > 2. I need to have custom clustering and classify results into certain > > classes only (i.e. only few very specific words in the search results). > > Like for example "Red", "Green", "Blue" etc... and not "hello World", > > "Known World", "green world" etc -(if you know what I mean here) - > > Where all these words in both Do and DoNot existing in the search > results. > > > > Please tell me how to achieve this. Perhaps Carrot/clustering is not > needed > > here and some other classifier is needed. So what to do here? > > > > Basically, I cannot receive 1 million results, then process them via > > PHP-Array to classify them as per need. The classification must be done > > here in solr only. > > > > Thanks > > > > -- > > Regards, > > Raheel Hasan > -- Regards, Raheel Hasan
Facet count for "others" after facet.limit
Hi, Is there anyway to use facet such that the results shows "Others" (or any default value) and show all the others? For example: on category_code count 6 1 false This will show top 6 different products counts divided into the categories. However, there are say 20 different categories and I want the rest of the counts to into "Others". so we have a total of 7 facet counts only: 6 categories and all the rest in "Others". Please let me know how to do this. thanks.. -- Regards, Raheel Hasan
Re: Facet count for "others" after facet.limit
Yea, I just thought about the calculation from [total results - all facet results]... But I wish there was a simple "Others" option as well ... Thanks anyway for your help. On Mon, Jun 10, 2013 at 8:20 PM, Jack Krupansky wrote: > Not directly for a field facet. Range and date facets do have the concept > of "other" to give you more details, but field facet doesn't have that. > > But, you can calculate that number easily - it is numFound minus the sum > of the facet counts for the field, minus "missing". > > Still, I agree that it would be nice to enable it directly, like > "facet.others=true". > > -- Jack Krupansky > > -Original Message- From: Raheel Hasan > Sent: Monday, June 10, 2013 10:56 AM > To: solr-user@lucene.apache.org > Subject: Facet count for "others" after facet.limit > > > Hi, > > Is there anyway to use facet such that the results shows "Others" (or any > default value) and show all the others? > > For example: > > on > category_**code > count > 6 > 1 > false > > This will show top 6 different products counts divided into the categories. > However, there are say 20 different categories and I want the rest of the > counts to into "Others". so we have a total of 7 facet counts only: 6 > categories and all the rest in "Others". > > Please let me know how to do this. thanks.. > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
