DataImportHandler ignoring most rows
Using: - apache-solr-1.3.0 - java 1.6 - tomcat 6 - sql server 2005 w/ JSQLConnect 4.0 driver I have a group table with 3007 rows. I have confirmed the key is unique with "select distinct id from group" and it returns 3007. When i re-index using http://host:port/solr/dataimport?command=full-import I only get 7 records indexed. Any insight into what is going on would be really great. A partial response: 1 7 0 I have other entities that index all the rows without issue. There are no errors in the logs. I am not using any Transformers (and most of my config is not changed from install) My schema.xml contains: key and field defs (not a full list of fields): data-config.xml
Re: DataImportHandler ignoring most rows
Upgraded to the tip from svn and still no love. - Original Message From: Erik Earle To: solr-user@lucene.apache.org Sent: Tuesday, August 18, 2009 3:16:47 PM Subject: DataImportHandler ignoring most rows Using: - apache-solr-1.3.0 - java 1.6 - tomcat 6 - sql server 2005 w/ JSQLConnect 4.0 driver I have a group table with 3007 rows. I have confirmed the key is unique with "select distinct id from group" and it returns 3007. When i re-index using http://host:port/solr/dataimport?command=full-import I only get 7 records indexed. Any insight into what is going on would be really great. A partial response: 1 7 0 I have other entities that index all the rows without issue. There are no errors in the logs. I am not using any Transformers (and most of my config is not changed from install) My schema.xml contains: key and field defs (not a full list of fields): data-config.xml
Solr Query help - sorting
Suppose I have a database of rectangle attributes that I have indexed in Solr and I want to get the top 10 widest or longest rectangles from one query. name | width | length A | 100| 10 B | 10 | 90 C | 80 | 10 ... My indexed fields are: name, width, length Is there a way to query so that my data returns the order: A, B, C
Re: Solr Query help - sorting
Is there a way to have the max_side field only in Solr ...as in a conditional copyField or something like that? I'd like to push as much of this into Solr as I can because the app and db that Solr is indexing are not really the best place to add this type of functionality. - Original Message From: Koji Sekiguchi To: solr-user@lucene.apache.org Sent: Tuesday, August 25, 2009 4:04:29 AM Subject: Re: Solr Query help - sorting Hi Erik Earle, Ahh, I read your mail too fast... Erik Hatcher's method should work. Thanks! Koji Erik Hatcher wrote: > You couldn't sort on a multiValued field though. > > I'd simply index a max_side field, and have the indexing client add a > single valued field with max(length,width) to it. Then sort on max_side. > > Erik > > On Aug 25, 2009, at 4:00 AM, Constantijn Visinescu wrote: > >> make a new multivalued field in your schema.xml, copy both width and >> length >> into that field, and then sort on that field ? >> >> On Tue, Aug 25, 2009 at 5:40 AM, erikea...@yahoo.com >> wrote: >> >>> Clever... but if more than one row adds up to the same value I may >>> get the >>> wrong order (like 50, 50 and 10, 90) >>> >>> I need a max function but the one provided only compares against a >>> constant. >>> >>> Sent from my iPhone >>> >>> On Aug 24, 2009, at 6:15 PM, Koji Sekiguchi wrote: >>> >>> How about using sum() FunctionQuery, via: >>> >>> http://localhost:8983/solr/select/?q=_val_%3A%22sum%28width%2Clength%29%22 >>> >>> >>> Thanks, >>> >>> Koji >>> >>> >>> Erik Earle wrote: >>> Suppose I have a database of rectangle attributes that I have >>> indexed in >>> Solr and I want to get the top 10 widest or longest rectangles from one >>> query. >>> >>> >>> name | width | length >>> A | 100| 10 >>> B | 10 | 90 >>> C | 80 | 10 >>> ... >>> >>> >>> My indexed fields are: name, width, length >>> >>> Is there a way to query so that my data returns the order: A, B, C >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> > >
Re: Solr Query help - sorting
I am indexing my data both through DataImportHandler and per transaction from JPA using @PostXXX listeners. UpdateRequestProcessor looks like exactly what I need I don't suppose there's a scriptable subclass available in 1.4 that is configured from schema.xml? :-) Thanks guys! - Original Message From: Erik Hatcher To: solr-user@lucene.apache.org Sent: Tuesday, August 25, 2009 11:26:58 AM Subject: Re: Solr Query help - sorting If you're using DataImportHandler, a custom (Java or script) transformer could do this. Also an UpdateProcessor could do it. But there is no conditional copyField capabilities otherwise. Keep in mind that pragmatically, if you're doing your own indexing code, why not have a line like this? :) max_side = (length > width) ? length : width Erik On Aug 25, 2009, at 2:20 PM, Erik Earle wrote: > Is there a way to have the max_side field only in Solr ...as in a > conditional copyField or something like that? > > I'd like to push as much of this into Solr as I can because the app > and db that Solr is indexing are not really the best place to add > this type of functionality. > > > > > > > > > > - Original Message > From: Koji Sekiguchi > To: solr-user@lucene.apache.org > Sent: Tuesday, August 25, 2009 4:04:29 AM > Subject: Re: Solr Query help - sorting > > Hi Erik Earle, > > Ahh, I read your mail too fast... Erik Hatcher's method should work. > > Thanks! > > Koji > > > Erik Hatcher wrote: >> You couldn't sort on a multiValued field though. >> >> I'd simply index a max_side field, and have the indexing client add a >> single valued field with max(length,width) to it. Then sort on >> max_side. >> >>Erik >> >> On Aug 25, 2009, at 4:00 AM, Constantijn Visinescu wrote: >> >>> make a new multivalued field in your schema.xml, copy both width and >>> length >>> into that field, and then sort on that field ? >>> >>> On Tue, Aug 25, 2009 at 5:40 AM, erikea...@yahoo.com >>> wrote: >>> >>>> Clever... but if more than one row adds up to the same value I may >>>> get the >>>> wrong order (like 50, 50 and 10, 90) >>>> >>>> I need a max function but the one provided only compares against a >>>> constant. >>>> >>>> Sent from my iPhone >>>> >>>> On Aug 24, 2009, at 6:15 PM, Koji Sekiguchi >>>> wrote: >>>> >>>> How about using sum() FunctionQuery, via: >>>> >>>> http://localhost:8983/solr/select/?q=_val_%3A%22sum%28width%2Clength%29%22 >>>> >>>> >>>> Thanks, >>>> >>>> Koji >>>> >>>> >>>> Erik Earle wrote: >>>> Suppose I have a database of rectangle attributes that I have >>>> indexed in >>>> Solr and I want to get the top 10 widest or longest rectangles >>>> from one >>>> query. >>>> >>>> >>>> name | width | length >>>> A | 100| 10 >>>> B | 10 | 90 >>>> C | 80 | 10 >>>> ... >>>> >>>> >>>> My indexed fields are: name, width, length >>>> >>>> Is there a way to query so that my data returns the order: A, B, C >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >> >> > > >
UpdateRequestProcessor config location
I've read through the wiki for this and it explains most everything except where in the solrconfig.xml the goes. I tried it at the top level but that doesn't seem to do anything. http://wiki.apache.org/solr/UpdateRequestProcessor
Re: UpdateRequestProcessor config location
I've implemented a fairly simple UpdateRequestProcessor much like the example here: http://wiki.apache.org/solr/UpdateRequestProcessor I attempted the below configuration in solrconfig.xml (like the above link shows) but nothing happens, no errorsnothing. Is this configuration supposed to be under the tag? list, of, comma, sep, values - Original Message From: Noble Paul നോബിള് नोब्ळ् To: solr-user@lucene.apache.org Sent: Thursday, August 27, 2009 9:57:54 PM Subject: Re: UpdateRequestProcessor config location could you provide more details on what exactly is that you have done? On Fri, Aug 28, 2009 at 7:08 AM, Erik Earle wrote: > > I've read through the wiki for this and it explains most everything except > where in the solrconfig.xml the goes. > > I tried it at the top level but that doesn't seem to do anything. > > http://wiki.apache.org/solr/UpdateRequestProcessor > > > > > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Config Relationship between MaxWarmingSearchers and StreamingUpdateSolrServer
My application updates the master index frequently, sometimes very frequently. Is there a good rule of thumb for configuring: 1) maxWarmingSearchers in the master 2) the SUSS thread pool size (and perhaps queue length) to match the server settings?
