Re: delta-import not giving updated records
You could try making a view for this query and then loading that view. This will give you simpler syntax in the data-config.xml file. You may find that the query you are trying is itself wrong. Using a view also lets you examine the data your are importing. Lance On 2/23/09, Noble Paul നോബിള് नोब्ळ् wrote: > > On Mon, Feb 23, 2009 at 2:09 PM, con wrote: > > > > HI > > > > I made that change of quotes and case sensitivity. But now i am getting > the > > below exception while running delta-import: > > > > Document # 1 > >at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:186) > >at > > > org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:143) > >at > > > org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:43) > >at > > > org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59) > >at > > > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:74) > >at > > > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285) > >at > > > org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:211) > >at > > > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:133) > >at > > > org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:359) > >at > > > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:388) > >at > > > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) > > Caused by: java.sql.SQLSyntaxErrorException: ORA-00918: column > ambiguously > > defined > > > >at > > > oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:91) > >at > oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:112) > >at > > > oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:173) > >at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:455) > >at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:413) > >at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:1030) > >at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:183) > >at > > oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:774) > >at > > > oracle.jdbc.driver.T4CStatement.executeMaybeDescribe(T4CStatement.java:849) > >at > > > oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1186) > >at > > > oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1770) > >at > oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1739) > >at > > > oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:298) > >at > > > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:179) > >... 10 more > > 13:48:22,538 ERROR [STDERR] 23 Feb, 2009 1:48:22 PM > > org.apache.solr.handler.dataimport.DataImporter doDeltaImport > > SEVERE: Delta Import Failed > > > > > > Also i tried replacing deltaQuery with deltaImportQuery. This is not > > throwing any exception but not updating the index also. I have put the > same > > query that is used in full import to do the delta import. > > > deltaImportQuery is not a replacement for deltaQuery > both hast to be present. 'deltaQuery' identifies the changed rows and > deltaImportQuery uses the values to import data > > > > > > Thanks > > con > > > > > > > > > > Shalin Shekhar Mangar wrote: > >> > >> 1. There is no closing quote in transformer="TemplateTransformer > >> 2. Attribute names are case-sensitive so it should be deltaQuery instead > >> of > >> deltaquery > >> > >> On Fri, Feb 20, 2009 at 6:48 PM, con wrote: > >> > >>> > >>> Hi alll > >>> > >>> I am trying to run delta-import. For this I am having the below > >>> data-config.xml > >>> > >>> > >>> >>> driver="oracle.jdbc.driver.OracleDriver" > >>> url="***" user="" password="*"/> > >>> > >>> >>> transformer="TemplateTransformer pk="USER_ID" > >>>query="select USERS.USER_ID, USERS.USER_NAME, > >>> USERS.CREATED_TIMESTAMP > >>> FROM USERS, CUSTOMERS where USERS.USER_ID = CUSTOMERS.USER_ID" > >>> > >>>deltaquery="select USERS.USER_ID, USERS.USER_NAME, > >>> USERS.CREATED_TIMESTAMP FROM USERS, CUSTOMERS where USERS.USER_ID = > >>> CUSTOMERS.USER_ID" > > >>> >>> /> > >>> > >>> > >>> > >>> > >>> But nothing is happening when i call > >>> http://localhost:8080/solr/users/dataimport?command=delta-import. > Whereas > >>> the dataimport.properties is getting updated with the time at which > >>> delta-import is run. > >>> > >>> Where as > >>> http://l
field collapsing in a nightly-build
I have seen field collapsing patch ( http://issues.apache.org/jira/browse/SOLR-236 ) has been market for Solr 1.5 but... Has anyone made it work with a recent nightly build ?? -- View this message in context: http://www.nabble.com/field-collapsing-in-a-nightly-build-tp22179010p22179010.html Sent from the Solr - User mailing list archive at Nabble.com.
Field Boosting with Alt Query
Hi, I am trying to achieve field boosting with alt query. I am giving wildcard in the query and it is working fine and field boosting is working with normal q query. I want to achieve both with q query. I am able to do wildcard search with q query but now I am not getting the field boosting happening. Please suggest. thanks, Amit Garg -- View this message in context: http://www.nabble.com/Field-Boosting-Code-tp22118997p22179260.html Sent from the Solr - User mailing list archive at Nabble.com.
Wildcard search with q query
Hi, I am trying to perform wildcard search using q query. e.g. If I give tes* as the query, it works fine and returns the results as expected. But if I give *tes*, then it throws an exception saying that we cant have wildcards in front of any string. Please suggest. Thanks, Amit Garg -- View this message in context: http://www.nabble.com/Wildcard-search-with-q-query-tp22179548p22179548.html Sent from the Solr - User mailing list archive at Nabble.com.
Field Boosting with wildcard
Hi, I am trying to achieve the field boosting in wild card search. If I do normal search, then this field boosting works fine but if I use wildcard in the search string, then it field boosting doesnt work. Please suggest. thanks, Amit garg -- View this message in context: http://www.nabble.com/Field-Boosting-with-wildcard-tp22179564p22179564.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Wildcard search with q query
Hi Amit, Leading wildcard searches are not allowing in Solr Out-of-the-box. There are other techniques to overcome this handicap. Search for "Leading Wildcards" in the user mailing list archives will return the necessary mail threads which might be pretty useful to you. -Kumar -Original Message- From: dabboo [mailto:ag...@sapient.com] Sent: Tuesday, February 24, 2009 4:04 PM To: solr-user@lucene.apache.org Subject: Wildcard search with q query Hi, I am trying to perform wildcard search using q query. e.g. If I give tes* as the query, it works fine and returns the results as expected. But if I give *tes*, then it throws an exception saying that we cant have wildcards in front of any string. Please suggest. Thanks, Amit Garg -- View this message in context: http://www.nabble.com/Wildcard-search-with-q-query-tp22179548p22179548.h tml Sent from the Solr - User mailing list archive at Nabble.com.
RE: Field Boosting with wildcard
Hi Amit, Wildcard searches do not support highlighting and rank boosting from Solr-1.3 onwards. -Kumar -Original Message- From: dabboo [mailto:ag...@sapient.com] Sent: Tuesday, February 24, 2009 4:06 PM To: solr-user@lucene.apache.org Subject: Field Boosting with wildcard Hi, I am trying to achieve the field boosting in wild card search. If I do normal search, then this field boosting works fine but if I use wildcard in the search string, then it field boosting doesnt work. Please suggest. thanks, Amit garg -- View this message in context: http://www.nabble.com/Field-Boosting-with-wildcard-tp22179564p22179564.h tml Sent from the Solr - User mailing list archive at Nabble.com.
any experiences with running SOLR on OpenJDK?
Hi all, does anyone have experience with running SOLR on OpenJDK 6.0? Any data points, positive or negative, would be appreciated. I am trying to decide whether to switch to OpenJDK on Debian Lenny, or whether to stick with the non-free JDK 5.0 for the time being. Best regards - Christian
Re: Field Boosting with wildcard
Hi, Thanks for the information. But can we achieve wildcards with q query instead of q.alt parameter. thanks, Amit dabboo wrote: > > Hi, > > I am trying to achieve the field boosting in wild card search. If I do > normal search, then this field boosting works fine but if I use wildcard > in the search string, then it field boosting doesnt work. > > Please suggest. > > thanks, > Amit garg > -- View this message in context: http://www.nabble.com/Field-Boosting-with-wildcard-tp22179564p22180669.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: any experiences with running SOLR on OpenJDK?
Christian Vogler wrote: Hi all, does anyone have experience with running SOLR on OpenJDK 6.0? Any data points, positive or negative, would be appreciated. I am trying to decide whether to switch to OpenJDK on Debian Lenny, or whether to stick with the non-free JDK 5.0 for the time being. Best regards - Christian I've done a lot of work for many months now using OpenJDK with both Solr and Lucene (both locally and on EC2) and I have not seen anything out of the ordinary. I wasn't running anything in production with it though - still, it was heavy use and testing. -- - Mark http://www.lucidimagination.com
Re: arcane queryParser parseException
> > : I went ahead and added it since it does not hurt anything to escape more > : things -- it just makes the final string ugly. > > : In 1.3 the escape method covered everything: > > H good call, i didn't realize the escape method had been so > blanket in 1.3. this way we protect people who were using it in 1.3 and > relied on it to protect them from the legacy ";" behavior. Thanks hoss and ryan. That explains why the error was new to us-- we upgraded to trunk from 1.3 release and this exception came from a solrj processed query that used to work.
show up every parameter in my dismax query
Hi Sorry I dont remember what is the parameter which show up every parameters stores in my solrconfig.xml file for the dismax query ? thanks a lot, -- View this message in context: http://www.nabble.com/show-up-every-parameter-in-my-dismax-query-tp22181063p22181063.html Sent from the Solr - User mailing list archive at Nabble.com.
problem in post request
Hi, I am facing problem when sending a post request with curl. All the special characters are messed up. Thanks, Ajay
Re: show up every parameter in my dismax query
sunnyfr wrote: Hi Sorry I dont remember what is the parameter which show up every parameters stores in my solrconfig.xml file for the dismax query ? thanks a lot, echoParams=all ? regards, Koji
Re: show up every parameter in my dismax query
yes thanks a lot Koji, Koji Sekiguchi-2 wrote: > > sunnyfr wrote: >> Hi >> >> Sorry I dont remember what is the parameter which show up every >> parameters >> stores in my solrconfig.xml file for the dismax query ? thanks a >> lot, >> >> >> > echoParams=all ? > > regards, > > Koji > > > > -- View this message in context: http://www.nabble.com/show-up-every-parameter-in-my-dismax-query-tp22181063p22184676.html Sent from the Solr - User mailing list archive at Nabble.com.
java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map;
Seeing this in the logs of an otherwise working solr instance. Commits are done automatically I believe every 10m or 1 docs. This is solr trunk (last updated last night) Any ideas? INFO: [] webapp=/solr path=/select params={fl=thingID,n_thingname,score&q=n_thingname:"Cornell+Dupree"^5+net_thingname:"Cornell+Dupree"^4+ne_thingname:"Cornell+Dupree"^2&wt=standard&fq=s_type:artist&rows=10&version=2.2} hits=2 status=0 QTime=37 Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onCommit INFO: SolrDeletionPolicy.onCommit: commits:num=2 commit{dir=/vol/solr/data/index,segFN=segments_2cy,version=1224560226691,generation=3058,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, segments_2cy, _2yp.tii, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yp.fdx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yr.tvd, _2yn_9.del, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yp.tvd] commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, segments_2cz, _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yt.tii, _2yt.frq, _2yp.tvd] Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit = 1224560226692 Feb 24, 2009 5:05:53 PM org.apache.solr.search.SolrIndexSearcher INFO: Opening searc...@25ddfb6a main Feb 24, 2009 5:05:53 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: end_commit_flush Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onInit INFO: SolrDeletionPolicy.onInit: commits:num=1 commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, segments_2cz, _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yt.tii, _2yt.frq, _2yp.tvd] Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit = 1224560226692 Feb 24, 2009 5:05:53 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map; at org.apache.solr.search.FastLRUCache.getStatistics(FastLRUCache.java:244) at org.apache.solr.search.FastLRUCache.toString(FastLRUCache.java:260) at java.lang.String.valueOf(String.java:2827) at java.lang.StringBuilder.append(StringBuilder.java:115) at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1645) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1147) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) Feb 24, 2009 5:05:53 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to searc...@25ddfb6a main Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={start=0&q=solr&rows=10} hits=0 status=0 QTime=2 Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={start=0&q=rocks&rows=10} hits=0 status=0 QTime=0 Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={q=stat
RE: Direct control over document position in search results
Hi Tolga, Here's a good place to start: http://wiki.apache.org/solr/QueryElevationComponent Steve On 2/23/2009 at 7:47 PM, Ercan, Tolga wrote: > I was wondering if there was any facility to directly manipulate search > results based on business criteria to place documents at a fixed > position in those results. For example, when I issue a query, the first > four results would be based on natural search relevancy, then the fifth > result would be based on the most relevant document when doctype:video > (if I had a doctype field of course), then results 6...* would resume > natural search relevancy? > > Or perhaps a variation on this, if the document where doctype:video > would appear at a fixed position or better... For example, if somebody > searched for "my widget video", there would be a relevant document at a > higher position than #5...
Collection distribution in a multicore environment
Hi fellow Solr fans, I'm setting up some collection distribution along with multicore solr . I'm using version 1.3 I have no problem with the snapshooter, since this can be set within each core in solrconfig.xml. My question is more about the rsyncd . The rsyncd-start creates a rsyncd.conf in the conf directory relative to where it lies , so what I did is copying bin/rsynd-start in each core directory: solr/ core1/ bin/ rsyncd-start conf/ rsyncd.conf core2/ - same thing - Then for each core, I launch a rsyncd : /../solr/core1/bin/rsyncd-start -p 18080 -d /../solr/core1/data/ This way, it can be stopped properly when I use (rsyncd-stop grabs the data from the conf/rsyncd.conf of the containing core). /../solr/core1/bin/rsyncd-stop The problem is I'm not very confortable with having one running deamon per core (each on a different port), plus a copy of each script inside each core. Is there any better way to set this up ? Cheers !! Jerome Eteve. -- Jerome Eteve. Chat with me live at http://www.eteve.net jer...@eteve.net
Re: sint vs integer
I believe the difference is that an sint is mangled in such a way that it will sort properly (i.e. during a range query) where as a regular integer is not. Since everything is stored as a string in lucene, sorting by numbers happens lexicographically which may not be the same as an ordinal sort. The sint datatype will ensure this ordinal sort by some mangling of the characters (hence if you look at an sint from the lucene index it looks like garbage). Hope that helps Amit On Mon, Feb 23, 2009 at 5:40 PM, Jonathan Haddad wrote: > What are the differences between using an sint and an integer, aside > from the range queries on sint? If I've indexed a field as an > integer, and I try to sort on it, will there be performance problem? > > About 1.5 million documents in index. > -- > Jonathan Haddad > http://www.rustyrazorblade.com >
warming question
Hey there, Is there any dynamic way to specify the queries to do the warming? I mean, not writing them hardcoded in solrconfig.xml but getting them from a database or from another file?? Thanks in advance -- View this message in context: http://www.nabble.com/warming-question-tp22187322p22187322.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map;
Most likely a jar issue. Did you clean before dist? The commons jar is probably out of whack with the solr jar - or something in that realm of thought. I'd fresh build and deploy again. -- - Mark http://www.lucidimagination.com Brian Whitman wrote: Seeing this in the logs of an otherwise working solr instance. Commits are done automatically I believe every 10m or 1 docs. This is solr trunk (last updated last night) Any ideas? INFO: [] webapp=/solr path=/select params={fl=thingID,n_thingname,score&q=n_thingname:"Cornell+Dupree"^5+net_thingname:"Cornell+Dupree"^4+ne_thingname:"Cornell+Dupree"^2&wt=standard&fq=s_type:artist&rows=10&version=2.2} hits=2 status=0 QTime=37 Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onCommit INFO: SolrDeletionPolicy.onCommit: commits:num=2 commit{dir=/vol/solr/data/index,segFN=segments_2cy,version=1224560226691,generation=3058,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, segments_2cy, _2yp.tii, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yp.fdx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yr.tvd, _2yn_9.del, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yp.tvd] commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, segments_2cz, _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yt.tii, _2yt.frq, _2yp.tvd] Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit = 1224560226692 Feb 24, 2009 5:05:53 PM org.apache.solr.search.SolrIndexSearcher INFO: Opening searc...@25ddfb6a main Feb 24, 2009 5:05:53 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: end_commit_flush Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onInit INFO: SolrDeletionPolicy.onInit: commits:num=1 commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, segments_2cz, _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yt.tii, _2yt.frq, _2yp.tvd] Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit = 1224560226692 Feb 24, 2009 5:05:53 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map; at org.apache.solr.search.FastLRUCache.getStatistics(FastLRUCache.java:244) at org.apache.solr.search.FastLRUCache.toString(FastLRUCache.java:260) at java.lang.String.valueOf(String.java:2827) at java.lang.StringBuilder.append(StringBuilder.java:115) at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1645) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1147) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) Feb 24, 2009 5:05:53 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to searc...@25ddfb6a main Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={start=0&q=solr&rows=10} hits=0 status=0 QTime=
Re: java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map;
i hit that one too! try: ant clean On Feb 24, 2009, at 12:08 PM, Brian Whitman wrote: Seeing this in the logs of an otherwise working solr instance. Commits are done automatically I believe every 10m or 1 docs. This is solr trunk (last updated last night) Any ideas? INFO: [] webapp=/solr path=/select params={fl=thingID,n_thingname,score&q=n_thingname:"Cornell +Dupree"^5+net_thingname:"Cornell+Dupree"^4+ne_thingname:"Cornell +Dupree"^2&wt=standard&fq=s_type:artist&rows=10&version=2.2} hits=2 status=0 QTime=37 Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onCommit INFO: SolrDeletionPolicy.onCommit: commits:num=2 commit{dir=/vol/solr/data/ index ,segFN = segments_2cy ,version=1224560226691,generation=3058,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, segments_2cy, _2yp.tii, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yp.fdx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yr.tvd, _2yn_9.del, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yp.tvd] commit{dir=/vol/solr/data/ index ,segFN = segments_2cz ,version=1224560226692,generation=3059,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, segments_2cz, _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yt.tii, _2yt.frq, _2yp.tvd] Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit = 1224560226692 Feb 24, 2009 5:05:53 PM org.apache.solr.search.SolrIndexSearcher INFO: Opening searc...@25ddfb6a main Feb 24, 2009 5:05:53 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: end_commit_flush Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onInit INFO: SolrDeletionPolicy.onInit: commits:num=1 commit{dir=/vol/solr/data/ index ,segFN = segments_2cz ,version=1224560226692,generation=3059,filenames=[_2yp.tvf, _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, segments_2cz, _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yt.tii, _2yt.frq, _2yp.tvd] Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit = 1224560226692 Feb 24, 2009 5:05:53 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NoSuchMethodError: org .apache .solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/ util/Map; at org.apache.solr.search.FastLRUCache.getStatistics(FastLRUCache.java: 244) at org.apache.solr.search.FastLRUCache.toString(FastLRUCache.java:260) at java.lang.String.valueOf(String.java:2827) at java.lang.StringBuilder.append(StringBuilder.java:115) at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java: 1645) at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1147) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor $Worker.runTask(ThreadPoolExecutor.java:885) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) Feb 24, 2009 5:05:53 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to searc...@25ddfb6a main Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=null path=null params={start=0&q=solr&rows=10} hits=0 status=0 QTime=2 Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=null pa
Re: Error with highlighter and UTF-8 chars?
Here you can see a manifestation of it when trying to highlight with ?q=daß − − − -Community" einfach nicht mehr wahrnimmt. Hätte mir am letzten Montag Nachmittag jemand gesagt, daß ich am Abend − recht, wenn er sagte, daß das wirklich wertvolle an Drupal schlichtweg seine (Entwickler- und Anwender-) − die Entstehungsgeschichte des Portals) auch dokumentiert worden, denn Ihr vermutet schon richtig, daß da You can see the "strong" tags each get offset one character more from where they are supposed to be. -Peter On Mon, Feb 23, 2009 at 8:24 AM, Peter Wolanin wrote: > We are using Solr trunk (1.4) - currently " nightly exported - yonik > - 2009-02-05 08:06:00" > > -Peter > > On Mon, Feb 23, 2009 at 8:07 AM, Koji Sekiguchi wrote: >> Jacob, >> >> What Solr version are you using? There is a bug in SolrHighlighter of Solr >> 1.3, >> you may want to look at: >> >> https://issues.apache.org/jira/browse/SOLR-925 >> https://issues.apache.org/jira/browse/LUCENE-1500 >> >> regards, >> >> Koji >> >> >> Jacob Singh wrote: >>> >>> Hi, >>> >>> We ran into a weird one today. We have a document which is written in >>> German and everytime we make a query which matches it, we get the >>> following: >>> >>> java.lang.StringIndexOutOfBoundsException: String index out of range: 2822 >>> at java.lang.String.substring(String.java:1935) >>> at >>> org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:274) >>> >>> >>> >From source diving it looks like Lucene's highlighter is trying to >>> subStr against an offset that is outside the bounds of the body field >>> which it is highlighting against. Running a fq against the ID of the >>> doucment returns it fine (because no highlighting is done) and I took >>> the body and tried to cut the first 2822 chars and while it is near >>> the end of the body, it is still in range. >>> >>> Here is the related code: >>> >>> startOffset = tokenGroup.matchStartOffset; >>> endOffset = tokenGroup.matchEndOffset; >>> tokenText = text.substring(startOffset, endOffset); >>> >>> >>> This leads me to believe there is some problem with mb string encoding >>> and Lucene's counting. >>> >>> Any ideas here? Tomcat is configured with UTF-8 btw. >>> >>> Best, >>> Jacob >>> >>> >>> >> >> > > > > -- > Peter M. Wolanin, Ph.D. > Momentum Specialist, Acquia. Inc. > peter.wola...@acquia.com > -- Peter M. Wolanin, Ph.D. Momentum Specialist, Acquia. Inc. peter.wola...@acquia.com
Re: arcane queryParser parseException
: > H good call, i didn't realize the escape method had been so : > blanket in 1.3. this way we protect people who were using it in 1.3 and : > relied on it to protect them from the legacy ";" behavior. : Thanks hoss and ryan. That explains why the error was new to us-- we : upgraded to trunk from 1.3 release and this exception came from a solrj : processed query that used to work. I would still suggest you (and everybody else) always specify "sort" and "defType" params anyway (even if just as defaults in your requestHandler declaration), just to be very clear and complete about what you expect (and to ensure this weird legacy behavior doesn't burn you if clients who *aren't* using SolrJ (or ClientUtils) query your port. -Hoss
Re: Error with highlighter and UTF-8 chars?
So - something in the highlighting code is counting bytes when it should be counting characters. Looks like a lucene bug, so I'm surprised others have not hit this before. Probably this is it: https://issues.apache.org/jira/browse/LUCENE-1500 -Peter On Tue, Feb 24, 2009 at 2:22 PM, Peter Wolanin wrote: > Here you can see a manifestation of it when trying to highlight with ?q=daß > > > − > > − > > − > > -Community" einfach nicht mehr wahrnimmt. > Hätte mir am letzten Montag Nachmittag jemand gesagt, daß > ich am Abend > > − > > recht, wenn er sagte, daß das wirklich wertvolle an > Drupal schlichtweg seine (Entwickler- und Anwender-) > > − > > die Entstehungsgeschichte des Portals) auch dokumentiert worden, denn > Ihr vermutet schon richtig, daß da > > > > > > > You can see the "strong" tags each get offset one character more from > where they are supposed to be. > > > -Peter > > > > On Mon, Feb 23, 2009 at 8:24 AM, Peter Wolanin > wrote: >> We are using Solr trunk (1.4) - currently " nightly exported - yonik >> - 2009-02-05 08:06:00" >> >> -Peter >> >> On Mon, Feb 23, 2009 at 8:07 AM, Koji Sekiguchi wrote: >>> Jacob, >>> >>> What Solr version are you using? There is a bug in SolrHighlighter of Solr >>> 1.3, >>> you may want to look at: >>> >>> https://issues.apache.org/jira/browse/SOLR-925 >>> https://issues.apache.org/jira/browse/LUCENE-1500 >>> >>> regards, >>> >>> Koji >>> >>> >>> Jacob Singh wrote: Hi, We ran into a weird one today. We have a document which is written in German and everytime we make a query which matches it, we get the following: java.lang.StringIndexOutOfBoundsException: String index out of range: 2822 at java.lang.String.substring(String.java:1935) at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:274) >From source diving it looks like Lucene's highlighter is trying to subStr against an offset that is outside the bounds of the body field which it is highlighting against. Running a fq against the ID of the doucment returns it fine (because no highlighting is done) and I took the body and tried to cut the first 2822 chars and while it is near the end of the body, it is still in range. Here is the related code: startOffset = tokenGroup.matchStartOffset; endOffset = tokenGroup.matchEndOffset; tokenText = text.substring(startOffset, endOffset); This leads me to believe there is some problem with mb string encoding and Lucene's counting. Any ideas here? Tomcat is configured with UTF-8 btw. Best, Jacob >>> >>> >> >> >> >> -- >> Peter M. Wolanin, Ph.D. >> Momentum Specialist, Acquia. Inc. >> peter.wola...@acquia.com >> > > > > -- > Peter M. Wolanin, Ph.D. > Momentum Specialist, Acquia. Inc. > peter.wola...@acquia.com > -- Peter M. Wolanin, Ph.D. Momentum Specialist, Acquia. Inc. peter.wola...@acquia.com
Re: Error with highlighter and UTF-8 chars?
Actually, looking at the Lucene source and the trace: java.lang.StringIndexOutOfBoundsException: String index out of range: 2822 at java.lang.String.substring(String.java:1765) at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:274) at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:313) at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:84) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) ... I see now that getBestTextFragments() takes in a token stream - and each token in this steam already has start/end positions set. So, the patch at LUCENE-1500 would mitigate the exception, but looks like the real bug is in Solr. -Peter On Tue, Feb 24, 2009 at 4:28 PM, Peter Wolanin wrote: > So - something in the highlighting code is counting bytes when it > should be counting characters. Looks like a lucene bug, so I'm > surprised others have not hit this before. Probably this is it: > https://issues.apache.org/jira/browse/LUCENE-1500 > > -Peter > > > On Tue, Feb 24, 2009 at 2:22 PM, Peter Wolanin > wrote: >> Here you can see a manifestation of it when trying to highlight with ?q=daß >> >> >> − >> >> − >> >> − >> >> -Community" einfach nicht mehr wahrnimmt. >> Hätte mir am letzten Montag Nachmittag jemand gesagt, daß >> ich am Abend >> >> − >> >> recht, wenn er sagte, daß das wirklich wertvolle an >> Drupal schlichtweg seine (Entwickler- und Anwender-) >> >> − >> >> die Entstehungsgeschichte des Portals) auch dokumentiert worden, denn >> Ihr vermutet schon richtig, daß da >> >> >> >> >> >> >> You can see the "strong" tags each get offset one character more from >> where they are supposed to be. >> >> >> -Peter >> >> >> >> On Mon, Feb 23, 2009 at 8:24 AM, Peter Wolanin >> wrote: >>> We are using Solr trunk (1.4) - currently " nightly exported - yonik >>> - 2009-02-05 08:06:00" >>> >>> -Peter >>> >>> On Mon, Feb 23, 2009 at 8:07 AM, Koji Sekiguchi wrote: Jacob, What Solr version are you using? There is a bug in SolrHighlighter of Solr 1.3, you may want to look at: https://issues.apache.org/jira/browse/SOLR-925 https://issues.apache.org/jira/browse/LUCENE-1500 regards, Koji Jacob Singh wrote: > > Hi, > > We ran into a weird one today. We have a document which is written in > German and everytime we make a query which matches it, we get the > following: > > java.lang.StringIndexOutOfBoundsException: String index out of range: 2822 > at java.lang.String.substring(String.java:1935) > at > org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:274) > > > >From source diving it looks like Lucene's highlighter is trying to > subStr against an offset that is outside the bounds of the body field > which it is highlighting against. Running a fq against the ID of the > doucment returns it fine (because no highlighting is done) and I took > the body and tried to cut the first 2822 chars and while it is near > the end of the body, it is still in range. > > Here is the related code: > > startOffset = tokenGroup.matchStartOffset; > endOffset = tokenGroup.matchEndOffset; > tokenText = text.substring(startOffset, endOffset); > > > This leads me to believe there is some problem with mb string encoding > and Lucene's counting. > > Any ideas here? Tomcat is configured with UTF-8 btw. > > Best, > Jacob > > > >>> >>> >>> >>> -- >>> Peter M. Wolanin, Ph.D. >>> Momentum Specialist, Acquia. Inc. >>> peter.wola...@acquia.com >>> >> >> >> >> -- >> Peter M. Wolanin, Ph.D. >> Momentum Specialist, Acquia. Inc. >> peter.wola...@acquia.com >> > > > > -- > Peter M. Wolanin, Ph.D. > Momentum Specialist, Acquia. Inc. > peter.wola...@acquia.com > -- Peter M. Wolanin, Ph.D. Momentum Specialist, Acquia. Inc. peter.wola...@acquia.com
Re: sint vs integer
On Mon, Feb 23, 2009 at 8:40 PM, Jonathan Haddad wrote: > What are the differences between using an sint and an integer, aside > from the range queries on sint? If I've indexed a field as an > integer, and I try to sort on it, will there be performance problem? Nope. If you don't need range queries, use "integer". Sorts work correctly on both because the sort code does not rely on the index order of the terms. -Yonik Lucene/Solr? http://www.lucidimagination.com
Re: Persistent, seemingly unfixable corrupt indices
Thanks for your answers Michael! I was using a pre-1.3 Solr build, but I've now upgraded to the 1.3 release, run the new CheckIndex shipped as part of the Lucene 2.4 dev build and I'm still getting the CorruptIndexException: docs out of order exceptions I'm afraid. Upon a fresh start, on newly Checked indices, I actually get a lot of Exceptions like: SEVERE: java.lang.RuntimeException: [was class org.mortbay.jetty.EofException] null at com.ctc.wstx.util.ExceptionUtil.throwRuntimeException(ExceptionUtil.java:18) at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:731) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3657) at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809) at org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:327) at org.apache.solr.handler.XmlUpdateRequestHandler.processUpdate(XmlUpdateRequestHandler.java:195) at org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:123) Before any CorruptIndexExceptions - could that be the root cause? Unfortunately the indices are large and contain confidential information; is there anything else I can do to identify where the problem is and why CheckIndex isn't catching it? Thanks James 2009/2/23 Michael McCandless > > Actually, even in 2.3.1, CheckIndex checks for docs-out-of-order both > within and across segments, so now I'm at a loss as to why it's not catching > your case. Any of these indexes small enough to post somewhere i could > access? > > Mike > > > James Brady wrote: > > Hi,My indices sometime become corrupted - normally when Solr has to be >> KILLed - these are not normally too much of a problem, as >> Lucene's CheckIndex tool can normally detect missing / broken segments and >> fix them. >> >> However, I now have a few indices throwing errors like this: >> >> INFO: [core4] webapp=/solr path=/update params={} status=0 QTime=2 >> Exception in thread "Thread-75" >> org.apache.lucene.index.MergePolicy$MergeException: >> org.apache.lucene.index.CorruptIndexException: docs out of order (1124 <= >> 1138 ) >> at >> >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271) >> Caused by: org.apache.lucene.index.CorruptIndexException: docs out of >> order >> (1124 <= 1138 ) >> at >> >> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502) >> at >> >> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456) >> at >> >> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425) >> at >> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389) >> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134) >> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109) >> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834) >> at >> >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240) >> >> and >> >> INFO: [core7] webapp=/solr path=/update params={} status=500 QTime=5457 >> Feb 22, 2009 12:14:07 PM org.apache.solr.common.SolrException log >> SEVERE: org.apache.lucene.index.CorruptIndexException: docs out of order >> (242 <= 248 ) >> at >> >> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:502) >> at >> >> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:456) >> at >> >> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:425) >> at >> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:389) >> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:134) >> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3109) >> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834) >> at >> >> org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:193) >> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1800) >> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1795) >> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1791) >> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2398) >> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1465) >> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1424) >> at >> >> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:278) >> >> >> CheckIndex reports these cores as being completely healthy, and yet I >> can't >> commit new documents in to them. >> >> Rebuilding indices isn't an option for me: is there any other way to fix >> this? If not, any ideas on what I can do to prevent it in the future? >> >> Many thanks, >> James >> > >
Re: java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map;
Yep, did ant clean, made sure all the solr-libs were current, no more exception. Thanks ryan & mark On Tue, Feb 24, 2009 at 1:47 PM, Ryan McKinley wrote: > i hit that one too! > > try: ant clean > > > > On Feb 24, 2009, at 12:08 PM, Brian Whitman wrote: > > Seeing this in the logs of an otherwise working solr instance. Commits are >> done automatically I believe every 10m or 1 docs. This is solr trunk >> (last updated last night) Any ideas? >> >> >> >> INFO: [] webapp=/solr path=/select >> >> params={fl=thingID,n_thingname,score&q=n_thingname:"Cornell+Dupree"^5+net_thingname:"Cornell+Dupree"^4+ne_thingname:"Cornell+Dupree"^2&wt=standard&fq=s_type:artist&rows=10&version=2.2} >> hits=2 status=0 QTime=37 >> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onCommit >> INFO: SolrDeletionPolicy.onCommit: commits:num=2 >> >> commit{dir=/vol/solr/data/index,segFN=segments_2cy,version=1224560226691,generation=3058,filenames=[_2yp.tvf, >> _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, segments_2cy, _2yp.tii, >> _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, >> _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, >> _2yo.tvd, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, >> _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, >> _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yr.prx, >> _2yo.tis, _2yp.fdt, _2yq.frq, _2yp.fdx, _2yq.fnm, _2yo.tvx, _2ys.tii, >> _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yr.tvd, >> _2yn_9.del, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yp.tvd] >> >> commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, >> _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, >> segments_2cz, >> _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, >> _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, >> _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, >> _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, >> _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, >> _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, >> _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, >> _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, >> _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, >> _2yt.tii, _2yt.frq, _2yp.tvd] >> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy >> updateCommits >> INFO: last commit = 1224560226692 >> Feb 24, 2009 5:05:53 PM org.apache.solr.search.SolrIndexSearcher >> INFO: Opening searc...@25ddfb6a main >> Feb 24, 2009 5:05:53 PM org.apache.solr.update.DirectUpdateHandler2 commit >> INFO: end_commit_flush >> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onInit >> INFO: SolrDeletionPolicy.onInit: commits:num=1 >> >> commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, >> _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, >> segments_2cz, >> _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, >> _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, >> _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, >> _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, >> _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, >> _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, >> _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, >> _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, >> _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, >> _2yt.tii, _2yt.frq, _2yp.tvd] >> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy >> updateCommits >> INFO: last commit = 1224560226692 >> Feb 24, 2009 5:05:53 PM org.apache.solr.common.SolrException log >> SEVERE: java.lang.NoSuchMethodError: >> >> org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map; >> at >> org.apache.solr.search.FastLRUCache.getStatistics(FastLRUCache.java:244) >> at org.apache.solr.search.FastLRUCache.toString(FastLRUCache.java:260) >> at java.lang.String.valueOf(String.java:2827) >> at java.lang.StringBuilder.append(StringBuilder.java:115) >> at >> org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1645) >> at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1147) >> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >> at >> >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) >> at >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) >> at java.lang.Thread.run(Thread.java:619) >> >> Feb 24, 2009 5:05:53 PM org.apache.so
Re: warming question
On Tue, Feb 24, 2009 at 11:45 PM, Marc Sturlese wrote: > > Hey there, > Is there any dynamic way to specify the queries to do the warming? I mean, > not writing them hardcoded in solrconfig.xml but getting them from a > database or from another file?? > Not right now. I had once wanted to do the same. There's an issue SOLR-784 open with a patch. -- Regards, Shalin Shekhar Mangar.
Re: java.lang.NoSuchMethodError: org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map;
please do a ant clean dist The signature of a method in ConsurrentLRUCache was altered from ConcurrentLRUCache.getLatestAccessedItems(long count) to ConcurrentLRUCache.getLatestAccessedItems(int count) On Wed, Feb 25, 2009 at 5:42 AM, Brian Whitman wrote: > Yep, did ant clean, made sure all the solr-libs were current, no more > exception. Thanks ryan & mark > > > > On Tue, Feb 24, 2009 at 1:47 PM, Ryan McKinley wrote: > >> i hit that one too! >> >> try: ant clean >> >> >> >> On Feb 24, 2009, at 12:08 PM, Brian Whitman wrote: >> >> Seeing this in the logs of an otherwise working solr instance. Commits are >>> done automatically I believe every 10m or 1 docs. This is solr trunk >>> (last updated last night) Any ideas? >>> >>> >>> >>> INFO: [] webapp=/solr path=/select >>> >>> params={fl=thingID,n_thingname,score&q=n_thingname:"Cornell+Dupree"^5+net_thingname:"Cornell+Dupree"^4+ne_thingname:"Cornell+Dupree"^2&wt=standard&fq=s_type:artist&rows=10&version=2.2} >>> hits=2 status=0 QTime=37 >>> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onCommit >>> INFO: SolrDeletionPolicy.onCommit: commits:num=2 >>> >>> commit{dir=/vol/solr/data/index,segFN=segments_2cy,version=1224560226691,generation=3058,filenames=[_2yp.tvf, >>> _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, segments_2cy, _2yp.tii, >>> _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, _2yr.tii, _2yr.nrm, _2ys.tvd, >>> _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, _2yn.tii, _2yn.fdt, _2yq.prx, >>> _2yo.tvd, _2yp.fnm, _2yo.tvf, _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, >>> _2yp.prx, _2yn.tis, _2yq.nrm, _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, >>> _2yq.tis, _2yo.fdx, _2yo.tii, _2yp.nrm, _2yq.tii, _2yr.frq, _2yr.prx, >>> _2yo.tis, _2yp.fdt, _2yq.frq, _2yp.fdx, _2yq.fnm, _2yo.tvx, _2ys.tii, >>> _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, _2ys.tis, _2yr.tvd, >>> _2yn_9.del, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, _2yp.tvd] >>> >>> commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, >>> _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, >>> segments_2cz, >>> _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, >>> _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, >>> _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, >>> _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, >>> _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, >>> _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, >>> _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, >>> _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, >>> _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, >>> _2yt.tii, _2yt.frq, _2yp.tvd] >>> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy >>> updateCommits >>> INFO: last commit = 1224560226692 >>> Feb 24, 2009 5:05:53 PM org.apache.solr.search.SolrIndexSearcher >>> INFO: Opening searc...@25ddfb6a main >>> Feb 24, 2009 5:05:53 PM org.apache.solr.update.DirectUpdateHandler2 commit >>> INFO: end_commit_flush >>> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy onInit >>> INFO: SolrDeletionPolicy.onInit: commits:num=1 >>> >>> commit{dir=/vol/solr/data/index,segFN=segments_2cz,version=1224560226692,generation=3059,filenames=[_2yp.tvf, >>> _2ys.prx, _2yo.fnm, _2yo.frq, _2yp.tvx, _2yn.fnm, _2yn_a.del, >>> segments_2cz, >>> _2yt.tvf, _2yp.tii, _2yt.tvd, _2yr.tis, _2ys.tvf, _2yp.frq, _2yn.tvf, >>> _2yr.tii, _2yr.nrm, _2ys.tvd, _2yn.fdx, _2yp.tis, _2yn.prx, _2yn.tvd, >>> _2yn.tii, _2yn.fdt, _2yq.prx, _2yo.tvd, _2yt.tvx, _2yp.fnm, _2yo.tvf, >>> _2yr.fdt, _2ys.frq, _2yn.nrm, _2yr.fdx, _2yp.prx, _2yn.tis, _2yq.nrm, >>> _2ys.tvx, _2ys.fnm, _2yo.fdt, _2yn.tvx, _2yq.tis, _2yo.fdx, _2yo.tii, >>> _2yp.nrm, _2yq.tii, _2yr.frq, _2yt.nrm, _2yr.prx, _2yo.tis, _2yp.fdt, >>> _2yq.frq, _2yt.fdx, _2yp.fdx, _2yt.fdt, _2yt.prx, _2yq.fnm, _2yo.tvx, >>> _2ys.tii, _2yt.fnm, _2yo.prx, _2yr.tvx, _2yn.frq, _2ys.nrm, _2yo.nrm, >>> _2ys.tis, _2yt.tis, _2yr.tvd, _2yr.tvf, _2yr.fnm, _2ys.fdx, _2ys.fdt, >>> _2yt.tii, _2yt.frq, _2yp.tvd] >>> Feb 24, 2009 5:05:53 PM org.apache.solr.core.SolrDeletionPolicy >>> updateCommits >>> INFO: last commit = 1224560226692 >>> Feb 24, 2009 5:05:53 PM org.apache.solr.common.SolrException log >>> SEVERE: java.lang.NoSuchMethodError: >>> >>> org.apache.solr.common.util.ConcurrentLRUCache.getLatestAccessedItems(J)Ljava/util/Map; >>> at >>> org.apache.solr.search.FastLRUCache.getStatistics(FastLRUCache.java:244) >>> at org.apache.solr.search.FastLRUCache.toString(FastLRUCache.java:260) >>> at java.lang.String.valueOf(String.java:2827) >>> at java.lang.StringBuilder.append(StringBuilder.java:115) >>> at >>> org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1645) >>> at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1147) >>> at java.util.concurrent.FutureTask$Sync.innerRun(Future
Re: warming question
I'm actually working on one for my company which parses our tomcat log files to obtain queries to feed as warming queries (since GET queries are the dominant source of queries) to the firstSearcher. I am not sure what the interface is in Solr 1.3, but in 1.2, I implemented the SolrEventListener interface and overrode the newSearcher method. If you look at the source for the default warmer, you should be able to construct a list of queries from a different source without much trouble. I might be able to send you some code if you need it. - Amit On Tue, Feb 24, 2009 at 10:15 AM, Marc Sturlese wrote: > > Hey there, > Is there any dynamic way to specify the queries to do the warming? I mean, > not writing them hardcoded in solrconfig.xml but getting them from a > database or from another file?? > Thanks in advance > -- > View this message in context: > http://www.nabble.com/warming-question-tp22187322p22187322.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Wildcards with q query parameter
Hi, can we achieve wildcard search with q query parameter. wildcard search works fine with q.alt parameter but not with q parameter. Also, I am trying to debug the QueryParser class for the same issue, but the control is not going in the right place in the eclipse. Hence, I am not able to debug that class file also. Please help. Thanks, Amit Garg -- View this message in context: http://www.nabble.com/Wildcards-with-q-query-parameter-tp22196748p22196748.html Sent from the Solr - User mailing list archive at Nabble.com.
Debugging Lucene Code
Hi, I am trying to debug code of QueryParser class and other related files. I have also taken the code of lucene from its SVN, but it is not going to the right control during debug. I wnated to know if I have taken the latest code and if not, from where I can take the code. Thansk, Amit Garg -- View this message in context: http://www.nabble.com/Debugging-Lucene-Code-tp22197796p22197796.html Sent from the Solr - User mailing list archive at Nabble.com.