Tokenizer question

[RL]

I could not find a solution to that in the documentation or the mailing list,
so here's my question.

I have files following the pattern: firstname_lastname_employeenumber.jpg

I'm able to search for the single terms firstname or lastname or the
employeenumber using a solr.PatternTokenizerFactory. Where I split at
underscore and dot.

But, now I also want to search for firstname_lastname or
lastname_employeenumber
Which does not work because the underscore was tokenized and is not part of
the indexed token anymore.


Any suggestions how to do that?

Thanks in advance.

RL



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Tokenizer-question-tp4016932.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR - To point multiple indexes in different folder

[ravi.n]

Erick,

Thanks for your response.
All the 7 folders are of same schema, i mean document structure is same. I
am not very sure how did customer get this data dump into different folders.
Now we have configured Solr with multicore, each core pointing to each
directory and using shards to get a single search response. Please suggest
is this right approach.

  







  


And now we should also configure solr for indexing new data from CSV file, i
am not sure how to configure this?

Regards,
Ravi



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SOLR-To-point-multiple-indexes-in-different-folder-tp4016640p4016946.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Urgent Help Needed: Solr Data import problem

[kunal sachdeva]

Hi,

I'm not getting this error while running in local machine. Please Help

Regards,
Kunal

On Tue, Oct 30, 2012 at 10:32 AM, Amit Nithian  wrote:

> This looks like a MySQL permissions problem and not a Solr problem.
> "Caused by: java.sql.SQLException: Access denied for user
> 'readonly'@'10.86.29.32'
> (using password: NO)"
>
> I'd advise reading your stack traces a bit more carefully. You should
> check your permissions or if you don't own the DB, check with your DBA
> to find out what user you should use to access your DB.
>
> - Amit
>
> On Mon, Oct 29, 2012 at 9:38 PM, kunal sachdeva
>  wrote:
> > Hi,
> >
> > I have tried using data-import in my local system. I was able to execute
> it
> > properly. but when I tried to do it unix server I got following error:-
> >
> >
> > INFO: Starting Full Import
> > Oct 30, 2012 9:40:49 AM
> > org.apache.solr.handler.dataimport.SimplePropertiesWriter
> > readIndexerProperties
> > WARNING: Unable to read: dataimport.properties
> > Oct 30, 2012 9:40:49 AM org.apache.solr.update.DirectUpdateHandler2
> > deleteAll
> > INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
> > Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy onInit
> > INFO: SolrDeletionPolicy.onInit: commits:num=1
> >
> >
> commit{dir=/opt/testsolr/multicore/core0/data/index,segFN=segments_1,version=1351490646879,generation=1,filenames=[segments_1]
> > Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
> > updateCommits
> > INFO: newest commit = 1351490646879
> > Oct 30, 2012 9:40:49 AM
> org.apache.solr.handler.dataimport.JdbcDataSource$1
> > call
> > INFO: Creating a connection for entity destination with URL:
> jdbc:mysql://
> > 172.16.37.160:3306/hpcms_db_new
> > Oct 30, 2012 9:40:50 AM org.apache.solr.common.SolrException log
> > SEVERE: Exception while processing: destination document :
> >
> SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
> > Unable to execute query: select name,id from hp_city Processing Document
> # 1
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)
> > at
> >
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
> > at
> >
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
> > at
> >
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)
> > Caused by: java.lang.RuntimeException:
> > org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> > execute query: select name,id from hp_city Processing Document # 1
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:621)
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)
> > ... 3 more
> > Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
> > Unable to execute query: select name,id from hp_city Processing Document
> # 1
> > at
> >
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
> > at
> >
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:253)
> > at
> >
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
> > at
> >
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
> > at
> >
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
> > at
> >
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
> > at
> >
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)
> > at
> >
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)
> > at
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:619)
> > ... 5 more
> > Caused by: java.sql.SQLException: Access denied for user
> > 'readonly'@'10.86.29.32'
> > (using password: NO)
> > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
> > at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3423)
> > at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:910)
> > at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:3923)
> > at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1273)
> > at
> > com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2031)
> > at com.mysql.jdbc.ConnectionImpl.(Con

Re: Urgent Help Needed: Solr Data import problem

[Dave Stuart]

It looks as though you have a password set on your unix server. you will need 
to either remove this or ti add the password into the connection string

e.g. readonly:[yourpassword]@'10.86.29.32'



>> 'readonly'@'10.86.29.32'
>> (using password: NO)"
On 30 Oct 2012, at 09:08, kunal sachdeva wrote:

> Hi,
> 
> I'm not getting this error while running in local machine. Please Help
> 
> Regards,
> Kunal
> 
> On Tue, Oct 30, 2012 at 10:32 AM, Amit Nithian  wrote:
> 
>> This looks like a MySQL permissions problem and not a Solr problem.
>> "Caused by: java.sql.SQLException: Access denied for user
>> 'readonly'@'10.86.29.32'
>> (using password: NO)"
>> 
>> I'd advise reading your stack traces a bit more carefully. You should
>> check your permissions or if you don't own the DB, check with your DBA
>> to find out what user you should use to access your DB.
>> 
>> - Amit
>> 
>> On Mon, Oct 29, 2012 at 9:38 PM, kunal sachdeva
>>  wrote:
>>> Hi,
>>> 
>>> I have tried using data-import in my local system. I was able to execute
>> it
>>> properly. but when I tried to do it unix server I got following error:-
>>> 
>>> 
>>> INFO: Starting Full Import
>>> Oct 30, 2012 9:40:49 AM
>>> org.apache.solr.handler.dataimport.SimplePropertiesWriter
>>> readIndexerProperties
>>> WARNING: Unable to read: dataimport.properties
>>> Oct 30, 2012 9:40:49 AM org.apache.solr.update.DirectUpdateHandler2
>>> deleteAll
>>> INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
>>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy onInit
>>> INFO: SolrDeletionPolicy.onInit: commits:num=1
>>> 
>>> 
>> commit{dir=/opt/testsolr/multicore/core0/data/index,segFN=segments_1,version=1351490646879,generation=1,filenames=[segments_1]
>>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
>>> updateCommits
>>> INFO: newest commit = 1351490646879
>>> Oct 30, 2012 9:40:49 AM
>> org.apache.solr.handler.dataimport.JdbcDataSource$1
>>> call
>>> INFO: Creating a connection for entity destination with URL:
>> jdbc:mysql://
>>> 172.16.37.160:3306/hpcms_db_new
>>> Oct 30, 2012 9:40:50 AM org.apache.solr.common.SolrException log
>>> SEVERE: Exception while processing: destination document :
>>> 
>> SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
>>> Unable to execute query: select name,id from hp_city Processing Document
>> # 1
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)
>>> Caused by: java.lang.RuntimeException:
>>> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
>>> execute query: select name,id from hp_city Processing Document # 1
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:621)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)
>>>... 3 more
>>> Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
>>> Unable to execute query: select name,id from hp_city Processing Document
>> # 1
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:253)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)
>>>at
>>> 
>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:619)
>>>... 5 more
>>> Caused by: java.sql.SQLException: Access denied for user
>>> 'readonly'@'10.86.29.32'
>>> (using password: NO)
>>>at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
>>>at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
>>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
>>>at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.j

Re: Urgent Help Needed: Solr Data import problem

[kunal sachdeva]

Hi,

This is my data-config file:-



  

  





 


  



and password is not null. and 10.86.29.32 is my unix server ip.

regards,
kunal

On Tue, Oct 30, 2012 at 2:42 PM, Dave Stuart  wrote:

> It looks as though you have a password set on your unix server. you will
> need to either remove this or ti add the password into the connection string
>
> e.g. readonly:[yourpassword]@'10.86.29.32'
>
>
>
> >> 'readonly'@'10.86.29.32'
> >> (using password: NO)"
> On 30 Oct 2012, at 09:08, kunal sachdeva wrote:
>
> > Hi,
> >
> > I'm not getting this error while running in local machine. Please Help
> >
> > Regards,
> > Kunal
> >
> > On Tue, Oct 30, 2012 at 10:32 AM, Amit Nithian 
> wrote:
> >
> >> This looks like a MySQL permissions problem and not a Solr problem.
> >> "Caused by: java.sql.SQLException: Access denied for user
> >> 'readonly'@'10.86.29.32'
> >> (using password: NO)"
> >>
> >> I'd advise reading your stack traces a bit more carefully. You should
> >> check your permissions or if you don't own the DB, check with your DBA
> >> to find out what user you should use to access your DB.
> >>
> >> - Amit
> >>
> >> On Mon, Oct 29, 2012 at 9:38 PM, kunal sachdeva
> >>  wrote:
> >>> Hi,
> >>>
> >>> I have tried using data-import in my local system. I was able to
> execute
> >> it
> >>> properly. but when I tried to do it unix server I got following error:-
> >>>
> >>>
> >>> INFO: Starting Full Import
> >>> Oct 30, 2012 9:40:49 AM
> >>> org.apache.solr.handler.dataimport.SimplePropertiesWriter
> >>> readIndexerProperties
> >>> WARNING: Unable to read: dataimport.properties
> >>> Oct 30, 2012 9:40:49 AM org.apache.solr.update.DirectUpdateHandler2
> >>> deleteAll
> >>> INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
> >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy onInit
> >>> INFO: SolrDeletionPolicy.onInit: commits:num=1
> >>>
> >>>
> >>
> commit{dir=/opt/testsolr/multicore/core0/data/index,segFN=segments_1,version=1351490646879,generation=1,filenames=[segments_1]
> >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
> >>> updateCommits
> >>> INFO: newest commit = 1351490646879
> >>> Oct 30, 2012 9:40:49 AM
> >> org.apache.solr.handler.dataimport.JdbcDataSource$1
> >>> call
> >>> INFO: Creating a connection for entity destination with URL:
> >> jdbc:mysql://
> >>> 172.16.37.160:3306/hpcms_db_new
> >>> Oct 30, 2012 9:40:50 AM org.apache.solr.common.SolrException log
> >>> SEVERE: Exception while processing: destination document :
> >>>
> >>
> SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
> >>> Unable to execute query: select name,id from hp_city Processing
> Document
> >> # 1
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)
> >>> Caused by: java.lang.RuntimeException:
> >>> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable
> to
> >>> execute query: select name,id from hp_city Processing Document # 1
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:621)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)
> >>>... 3 more
> >>> Caused by:
> org.apache.solr.handler.dataimport.DataImportHandlerException:
> >>> Unable to execute query: select name,id from hp_city Processing
> Document
> >> # 1
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:253)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)
> >>>at
> >>>
> >>
> org.apache.solr.handler.dataimport.DocB

Unable to build trunk

[Markus Jelsma]

Hi,

Since yesterday we're unable to build trunk and also a clean check out from 
trunk. We can compile the sources but not the example or dist.

It hangs on resolve and after a while prints the following:

resolve:

[ivy:retrieve] 
[ivy:retrieve] :: problems summary ::
[ivy:retrieve]  WARNINGS
[ivy:retrieve]  module not found: 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]   local: tried
[ivy:retrieve]
/home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
[ivy:retrieve]-- artifact 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
[ivy:retrieve]
/home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
[ivy:retrieve]   shared: tried
[ivy:retrieve]
/home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
[ivy:retrieve]-- artifact 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
[ivy:retrieve]
/home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
[ivy:retrieve]   public: tried
[ivy:retrieve]
http://repo1.maven.org/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
[ivy:retrieve]   sonatype-releases: tried
[ivy:retrieve]
http://oss.sonatype.org/content/repositories/releases/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
[ivy:retrieve]   working-chinese-mirror: tried
[ivy:retrieve]
http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
[ivy:retrieve]-- artifact 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
[ivy:retrieve]
http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.jar
[ivy:retrieve]  ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve]  ::
[ivy:retrieve]  :: 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4: not found
[ivy:retrieve]  ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve]  impossible to acquire lock for 
com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
[ivy:retrieve] 
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/home/markus/src/solr/trunk/solr/build.xml:336: The following error occurred 
while executing this line:
/home/markus/src/solr/trunk/solr/common-build.xml:345: The following error 
occurred while executing this line:
/home/markus/src/solr/trunk/solr/common-build.xml:388: The following error 
occurred while executing this line:
/home/markus/src/solr/trunk/lucene/common-build.xml:316: impossible to resolve 
dependencies:
resolve failed - see output for details

Total time: 18 minutes 19 seconds

As you can see, it's hanging a long time before it fails. Any ideas?

Thanks,
Markus

Re: hot shard concept

[Dmitry Kan]

Hi Shawn,

Thanks for sharing your story. Let me get it right:

How do you keep the incremental shard slim enough over time, do you
periodically redistribute the documents from it onto cold shards? If yes,
how technically you do it: the Lucene low-level way or Solr / SolrJ way?

-dmitry

On Mon, Oct 29, 2012 at 7:17 PM, Shawn Heisey  wrote:

> On 10/29/2012 7:55 AM, Dmitry Kan wrote:
>
>> Hi everyone,
>>
>> at this year's Berlin Buzz words conference someone (sematext?) have
>> described a technique of a hot shard. The idea is to have a slim shard to
>> maximize the update throughput during a day (when millions of docs need to
>> be posted) and make sure the indexed documents are immediately searchable.
>> In the end of the day the day's documents are moved to cold shards. If I'm
>> not mistaken, this was implemented for ElasticSearch. I'm currently
>> implementing something similar (but pretty tailored to our logical
>> sharding
>> use case) for Solr (3.x). The feature set looks roughly like this:
>>
>> 1) front end solr (query router) is aware of the hot shard: it directs the
>> incoming queries to the hot and "cold" shards.
>> 2) new incoming documents are directed first to the hot shard and then
>> periodically (like once a day or once a week) moved over to the closest in
>> time cold shard. And for that...
>> 3) hot shard index is being partitioned low level using Lucene's
>> IndexReader / IndexWriter with the implementation based on [1], [2] and
>> customized to logical (time-based) sharding.
>>
>>
>> The question is: is doing index partitioning low-level a good way of
>> implementing the hot shard concept? That is, is there anything better
>> operationally-wise from the point of view of disaster recovery / search
>> cluster support? Am I missing some obvious SOLR-ish solution?
>> Doing instead the periodical hot shard cleaning and re-posting its source
>> documents to the closest cold shard is less modular and hence more
>> complicated operationally for us.
>>
>> Please let me know, if you need more details or if the problem isn't clear
>> enough. Thanks.
>>
>> [1]
>> http://blog.foofactory.fi/**2008/01/regenerating-equally-**
>> sized-shards-from.html
>> [2] 
>> https://github.com/HON-**Khresmoi/hash-based-index-**splitter
>>
>
> This is exactly how I set up my indexing, been that way since early 2010
> when we first started using Solr 1.4.0.  Now we are on 3.5 and an upgrade
> to 4.1 (branch_4x) is in the works.  Coming from terminology used in our
> previous search product, we call the hot shard an "incremental" shard.  My
> SolrJ indexing application takes care of all management of which documents
> are in the incremental and which documents are in the large shards.  We
> call the large ones "static" shards because deletes and the occasional
> reinsert are the only updates that they receive, except for the daily
> distribute process.
>
> We don't do anything to send queries to the hot shard "first" ... it is
> simply listed first in the shards parameter on what we call the broker
> core.  Average response time on the incremental is single digit, the other
> shards average at about 30 to 40 milliseconds. Median numbers (SOLR-1972
> patch) are much better.
>
> Thanks,
> Shawn
>
>

Are there any limitations on multi-value field joins?

[Steven Livingstone Pérez]

Hi - I've done quite a bit of Googling and reading but can't find a definitive 
answer to this.
I would like to have a list of key data rows each with a unique id and some 
data.
datarow1 a b cdatarow2 x y zdatarow3 m n o...
I'd then like to have other rows that point to one or more of they data rows 
that have a multi-valued field that can contain one or many of the unique id's 
above.
User1 datarow1, datarow2, datarow3 etcUser2 datarow4, datarow21, datarow43 
etc...
Then i will join from the user1 row to the data row.
My question is simply are there *any* limitation on doing this kind of join? I 
believe there are some geo-spatial issues and sorting (i don't need to sort on 
the id) but before i jump fully into this approach i've like to understand 
anything i may run into - or whether it is better to have them as individual 
rows and join them that way.
many thanks,/steven   

RE: Query regarding solr search

[Leena Jawale]

Hi,

I understand that in the solr search considering the field type as text_en and 
not as int for the fields.
So how do I convert the field type of a particular field in solr XML as int so 
that I can operate that
field for range queries in solr??

Thanks,
Leena Jawale

From: Leena Jawale
Sent: Tuesday, October 30, 2012 12:18 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: Query regarding solr search

Hi,

I have created Solr XML data source. And on that I am working on less than 
operator.
I tried q=SerialNo:[ * TO 500 ].But It is showing records having SerialNo=1000. 
Could you help me this.?


Thanks,
Leena Jawale


The contents of this e-mail and any attachment(s) may contain confidential or 
privileged information for the intended recipient(s). Unintended recipients are 
prohibited from taking action on the basis of information in this e-mail and 
using or disseminating the information, and must notify the sender and delete 
it from their system. L&T Infotech will not accept responsibility or liability 
for the accuracy or completeness of, or the presence of any virus or disabling 
code in this e-mail"

Re: Query regarding solr search

[Gora Mohanty]

On 30 October 2012 16:57, Leena Jawale  wrote:
> Hi,
>
> I understand that in the solr search considering the field type as text_en 
> and not as int for the fields.
> So how do I convert the field type of a particular field in solr XML as int 
> so that I can operate that
> field for range queries in solr??
[...]

The field type is defined in the Solr schema.xml. The field
name in the Solr XML used to index to Solr must match this.

Regards,
Gora

Running Solr Core/ Tika on Azure

[Aloke Ghoshal]

Hi,

Looking for feedback on running Solr Core/ Tika parsing engine on Azure.
There's one offering for Solr within Azure from Lucid works. This offering
however doesn't mention Tika.

We are looking at options to make content from files (doc, excel, pdfs,
etc.) stored within Azure storage search-able. And whether the parser could
run against our Azure store directly to index the content. The other option
could be to write a separate connector that streams in the files. Let me
know if you have experience along these lines.

Regards,
Aloke

Re: row.get() in script transformer adds square brackets [] to string value

[Radek Zajkowski]

Thanks and sorry, forgot to include XML

In data config:



The Xml node:



Is the the CDATA section causing this?

Radek.

On Mon, Oct 29, 2012 at 9:10 PM, Jack Krupansky wrote:

> Sounds like it is multivalued - the square brackets indicate an array.
>
> -- Jack Krupansky
>
> -Original Message- From: Radek Zajkowski
> Sent: Monday, October 29, 2012 8:37 PM
> To: solr-user@lucene.apache.org
> Subject: row.get() in script transformer adds square brackets [] to string
> value
>
>
> Hi all,
>
> would you know why I get (notice square brackets)
>
> [1969 Harley Davidson Ultimate Chopper]
>
> not
>
> 1969 Harley Davidson Ultimate Chopper
>
> when calling
>
> var description = row.get("ProductName").**toString();
>
> in a script transformer?
>
> Thank you,
>
> Radek.
>

Re: Jetty / Solr memory consumption

[Erick Erickson]

I just skimmed your post, but have you seen:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

MMapDirectory may be giving you a false sense of how much physical
memory is actually being used.

Best
Erick

On Mon, Oct 29, 2012 at 1:59 PM, Nicolai Scheer
 wrote:
> Hi again!
>
> On 29 October 2012 18:39, Nicolai Scheer  wrote:
>> Hi!
>>
>> We're currently facing a strange memory issue we can't explain, so I'd
>> like to kindly ask if anyone is able to shed a light an the behavour
>> we encounter.
>>
>> We use a Solr 3.5 instance on a Windows Server 2008 machine equipped
>> with 16GB of ram.
>> The index uses 8 cores, 10 million documents, disk size of 180 GB in total.
>> The machine is only used for searches, text extraction is done on another 
>> box.
> [...]
>
> I should add which java version we're using:
>
> java -version
> java version "1.6.0_23"
> Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
> Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
>
> Greetings
>
> Nico

Re: Running Solr Core/ Tika on Azure

[Radek Zajkowski]

I've been able to run Solr on Azure with this

https://github.com/MSOpenTech/Windows-Azure-Solr

Had to do some modification as we didn't need multiple slaves etc. but
otherwise this was our jumping off point.

If your Azure instance can mount the drive with files than you should be
able to expose it to Tika. I don't think that it will be any different than
running Solr on Azure and indexing DB or XML.

Above project "boostraps" itself with all of the Java and Solr files it
needs to run and starts Solr using bundled in Jetty web server, so as long
as you have Tika in your libs and a configured handler you should be able
to use it.

Radek.

On Tue, Oct 30, 2012 at 4:31 AM, Aloke Ghoshal  wrote:

> Hi,
>
> Looking for feedback on running Solr Core/ Tika parsing engine on Azure.
> There's one offering for Solr within Azure from Lucid works. This offering
> however doesn't mention Tika.
>
> We are looking at options to make content from files (doc, excel, pdfs,
> etc.) stored within Azure storage search-able. And whether the parser could
> run against our Azure store directly to index the content. The other option
> could be to write a separate connector that streams in the files. Let me
> know if you have experience along these lines.
>
> Regards,
> Aloke
>

Re: Solr4.0 / SolrCloud queries

[Erick Erickson]

In addition to Shawn's comments, you might want to see:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

Lucene's use of MMapDirectory can mislead you when looking at
memory usage

Best
Erick

On Mon, Oct 29, 2012 at 5:59 PM, Shawn Heisey  wrote:
> On 10/29/2012 3:26 PM, shreejay wrote:
>>
>> I am trying to run two SolrCloud with 3 and 2 shards respectively (lets
>> say
>> Cloud3shards and Clouds2Shards). All servers are identical with 18GB Ram
>> (16GB assigned for Java).
>
>
> This bit right here sets off warning bells right away.  You're only leaving
> 2GB of RAM for the OS to cache your index, which you later say is 50GB.
> It's impossible for me to give you a precise figure, but I would expect that
> with an index that size, you'd want to have at least 20GB of free memory,
> and if you can have 50GB or more of free memory after the OS and Java take
> their chunk, Solr would have truly excellent performance.  As it is now,
> your performance will be terrible, which probably explains all your issues.
>
> It seems highly unlikely that you would have queries complex enough that you
> actually do need to allocate 16GB of RAM to Java.  Also, requesting large
> numbers of documents (the 5000 and 2 numbers you mentioned) is slow, and
> compounded in a cloud (distributed) index.  Solr is optimized for a small
> number of results.
>
> First recommendation for fixing things: get more memory.  32GB would be a
> good starting point, 64GB would be better, so that the entire index will be
> able to fit in OS cache memory.  If you expect your index to grow at all,
> plan accordingly.
>
> Second recommendation, whether or not you get more actual memory: Lower the
> memory that Java is using, and configure some alternate memory management
> options for Java.  Solr does have caching capability, but it is highly
> specialized.  For general index caching, the OS does a far better job, and
> it needs free memory in order to accomplish it.  Here's some commandline
> options for Java that I passed along to someone else on this list:
>
> -Xmx4096M -Xms4096M -XX:NewRatio=1 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:+CMSParallelRemarkEnabled
>
> http://www.petefreitag.com/articles/gctuning/
>

Re: SolrCloud and distributed search

[Erick Erickson]

you can certainly save one hop by sending the update request to
the shard leader (how are you sending things BTW, SolrJ?).

But at this point that hasn't been done, see:
https://issues.apache.org/jira/browse/SOLR-3154

It's certainly on the roadmap, but for now it works functionally
so there are other first to fry

FWIW,
Erick

On Mon, Oct 29, 2012 at 6:12 PM, Bill Au  wrote:
> Do updates always start at the shard leader first?  If so one can save one
> internal request by only sending updates to the shard leader.  I am
> assuming that when the shard leader is down, SolrJ's CloudSolrServer is
> smart enough to use the newly elected shard leader after a failover has
> occurred.  Am I correct?
>
> Bill
>
> On Fri, Oct 26, 2012 at 11:42 AM, Tomás Fernández Löbbe <
> tomasflo...@gmail.com> wrote:
>
>> If you are going to use SolrJ, CloudSolrServer is even better than a
>> round-robin load balancer for indexing, because it will send the documents
>> straight to the shard leader (you save one internal request). If not,
>> round-robin should be fine.
>>
>> Tomás
>>
>> On Fri, Oct 26, 2012 at 12:27 PM, Bill Au  wrote:
>>
>> > I am thinking of using a load balancer for both indexing and querying to
>> > spread both the indexing and querying load across all the machines.
>> >
>> > Bill
>> >
>> > On Fri, Oct 26, 2012 at 10:48 AM, Tomás Fernández Löbbe <
>> > tomasflo...@gmail.com> wrote:
>> >
>> > > You should still use some kind of load balancer for searches, unless
>> you
>> > > use the CloudSolrServer (SolrJ) which includes the load balancing.
>> > > Tomás
>> > >
>> > > On Fri, Oct 26, 2012 at 11:46 AM, Erick Erickson <
>> > erickerick...@gmail.com
>> > > >wrote:
>> > >
>> > > > Yes, I think SolrCloud makes sense with a single shard for exactly
>> > > > this reason, NRT and multiple replicas. I don't know how you'd get
>> NRT
>> > > > on multiple machines without it.
>> > > >
>> > > > But do be aware of: https://issues.apache.org/jira/browse/SOLR-3971
>> > > > "A collection that is created with numShards=1 turns into a
>> > > > numShards=2 collection after starting up a second core and not
>> > > > specifying numShards."
>> > > >
>> > > > Erick
>> > > >
>> > > > On Fri, Oct 26, 2012 at 10:14 AM, Bill Au 
>> wrote:
>> > > > > I am currently using one master with multiple slaves so I do have
>> > high
>> > > > > availability for searching now.
>> > > > >
>> > > > > My index does fit on a single machine and a single query does not
>> > take
>> > > > too
>> > > > > long to execute.  But I do want to take advantage of high
>> > availability
>> > > of
>> > > > > indexing and real time replication.  So it looks like I can set up
>> > > > > SolrCloud with only 1 shard (ie numShards=1).
>> > > > >
>> > > > > In this case is SolrCloud still using distributed search behind the
>> > > > > screen?  Will MoreLikeThis work?
>> > > > >
>> > > > > Does using SolrCloud with only 1 shard make any sense at all?
>> > > > >
>> > > > > Bill
>> > > > >
>> > > > > On Thu, Oct 25, 2012 at 4:29 PM, Tomás Fernández Löbbe <
>> > > > > tomasflo...@gmail.com> wrote:
>> > > > >
>> > > > >> It also provides high availability for indexing and searching.
>> > > > >>
>> > > > >> On Thu, Oct 25, 2012 at 4:43 PM, Bill Au 
>> > wrote:
>> > > > >>
>> > > > >> > So I guess one would use SolrCloud for the same reasons as
>> > > distributed
>> > > > >> > search:
>> > > > >> >
>> > > > >> > When an index becomes too large to fit on a single system, or
>> > when a
>> > > > >> single
>> > > > >> > query takes too long to execute.
>> > > > >> >
>> > > > >> > Bill
>> > > > >> >
>> > > > >> > On Thu, Oct 25, 2012 at 3:38 PM, Shawn Heisey <
>> s...@elyograg.org>
>> > > > wrote:
>> > > > >> >
>> > > > >> > > On 10/25/2012 1:29 PM, Bill Au wrote:
>> > > > >> > >
>> > > > >> > >> Is SolrCloud using distributed search behind the scene?  Does
>> > it
>> > > > have
>> > > > >> > the
>> > > > >> > >> same limitations (for example, doesn't support MoreLikeThis)
>> > > > >> distributed
>> > > > >> > >> search has?
>> > > > >> > >>
>> > > > >> > >
>> > > > >> > > Yes and yes.
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>>

Re: SOLR - To point multiple indexes in different folder

[Erick Erickson]

Until you nail down what the user did, this is may cause
problems. A sharded system assumes that the unique IDs
 in your schema exists on one and only one shard,
otherwise you'll be getting multiple copies of the docs.

And you've only shown a multi-core setup, NOT a sharded
setup. You need to define a searchhandler in solrconfig.xml
similar to a requestHandler and provide the shards as
defaults.

I don't have the reference close to hand, but you should be able
to find it with some searching. Beware the "recursion" problem
that you'll see referenced. Last I knew you can't configure your
shards in the default search handler, since that's the one that
gets the sub-requests for all your nodes

Best
Erick


On Tue, Oct 30, 2012 at 5:01 AM, ravi.n  wrote:
> Erick,
>
> Thanks for your response.
> All the 7 folders are of same schema, i mean document structure is same. I
> am not very sure how did customer get this data dump into different folders.
> Now we have configured Solr with multicore, each core pointing to each
> directory and using shards to get a single search response. Please suggest
> is this right approach.
>
>   
> 
> 
> 
> 
> 
> 
> 
>   
> 
>
> And now we should also configure solr for indexing new data from CSV file, i
> am not sure how to configure this?
>
> Regards,
> Ravi
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/SOLR-To-point-multiple-indexes-in-different-folder-tp4016640p4016946.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Running Solr Core/ Tika on Azure

[Aloke Ghoshal]

Thanks Radek. I have this as this one on my list as one option to be tried
out.

Regards,
Aloke

On Tue, Oct 30, 2012 at 5:26 PM, Radek Zajkowski  wrote:

> I've been able to run Solr on Azure with this
>
> https://github.com/MSOpenTech/Windows-Azure-Solr
>
> Had to do some modification as we didn't need multiple slaves etc. but
> otherwise this was our jumping off point.
>
> If your Azure instance can mount the drive with files than you should be
> able to expose it to Tika. I don't think that it will be any different than
> running Solr on Azure and indexing DB or XML.
>
> Above project "boostraps" itself with all of the Java and Solr files it
> needs to run and starts Solr using bundled in Jetty web server, so as long
> as you have Tika in your libs and a configured handler you should be able
> to use it.
>
> Radek.
>
> On Tue, Oct 30, 2012 at 4:31 AM, Aloke Ghoshal 
> wrote:
>
> > Hi,
> >
> > Looking for feedback on running Solr Core/ Tika parsing engine on Azure.
> > There's one offering for Solr within Azure from Lucid works. This
> offering
> > however doesn't mention Tika.
> >
> > We are looking at options to make content from files (doc, excel, pdfs,
> > etc.) stored within Azure storage search-able. And whether the parser
> could
> > run against our Azure store directly to index the content. The other
> option
> > could be to write a separate connector that streams in the files. Let me
> > know if you have experience along these lines.
> >
> > Regards,
> > Aloke
> >
>

Subqueries... using results from the main query

[Gustav]

Hello everyone,

i would like to know if it is possible to make a SUBQUERY inside my query
considering a field's current result of the main query in this subquery, 
Let me try to give a better explaination

I Have a simple query, something like this: /?q="1234"&qf=product_id, this
returns to me several fields, including:

14114 (this is the id of the related
product)
14115

I want to know if, in this same query, is possible to return the products
wich id are 14114 and 14115, instead of the normal results...

Thanks in advance, 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Subqueries-using-results-from-the-main-query-tp4017000.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Unable to build trunk

[Erick Erickson]

Not sure if it's relevant, but sometimes the ivy caches are wonky. Try
deleting (on OS X) ~/.ivy2 recursively and building again? Of course
your next build will download a bunch of jars...

FWIW,
Erick

On Tue, Oct 30, 2012 at 5:38 AM, Markus Jelsma
 wrote:
> Hi,
>
> Since yesterday we're unable to build trunk and also a clean check out from 
> trunk. We can compile the sources but not the example or dist.
>
> It hangs on resolve and after a while prints the following:
>
> resolve:
>
> [ivy:retrieve]
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve]  WARNINGS
> [ivy:retrieve]  module not found: 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]   local: tried
> [ivy:retrieve]
> /home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
> [ivy:retrieve]-- artifact 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
> [ivy:retrieve]
> /home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
> [ivy:retrieve]   shared: tried
> [ivy:retrieve]
> /home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
> [ivy:retrieve]-- artifact 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
> [ivy:retrieve]
> /home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
> [ivy:retrieve]   public: tried
> [ivy:retrieve]
> http://repo1.maven.org/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
> [ivy:retrieve]   sonatype-releases: tried
> [ivy:retrieve]
> http://oss.sonatype.org/content/repositories/releases/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
> [ivy:retrieve]   working-chinese-mirror: tried
> [ivy:retrieve]
> http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
> [ivy:retrieve]-- artifact 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
> [ivy:retrieve]
> http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.jar
> [ivy:retrieve]  ::
> [ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
> [ivy:retrieve]  ::
> [ivy:retrieve]  :: 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4: not found
> [ivy:retrieve]  ::
> [ivy:retrieve]  ERRORS
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]  impossible to acquire lock for 
> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> [ivy:retrieve]
> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>
> BUILD FAILED
> /home/markus/src/solr/trunk/solr/build.xml:336: The following error occurred 
> while executing this line:
> /home/markus/src/solr/trunk/solr/common-build.xml:345: The following error 
> occurred while executing this line:
> /home/markus/src/solr/trunk/solr/common-build.xml:388: The following error 
> occurred while executing this line:
> /home/markus/src/solr/trunk/lucene/common-build.xml:316: impossible to 
> resolve dependencies:
> resolve failed - see output for details
>
> Total time: 18 minutes 19 seconds
>
> As you can see, it's hanging a long time before it fails. Any ideas?
>
> Thanks,
> Markus

Re: Doc Transformer to remove document from the response

[eksdev]

I am not disputing that, the most general way to do it is to exclude hits just 
like you said. 
Was just thinking aloud about my particular use case, where I do not care about 
pagination.

Thanks for your comments, 
e.


On Oct 29, 2012, at 10:58 PM, Chris Hostetter  wrote:

> 
> : I did not look where pagination happens, but it looks like
> : DocTransform gets applied at the very end (response writer), which in
> : turn means pagination is not an issue , just soma pages might get
> : shorter due to this additional filtering, but that is quite ok for me.
> 
> it depends on what you mean by "not an issue" ... i would argue that if a 
> client asks for the first 10 matches, and you return a numFound of 678 but 
> only give back 8 matches (because you have "excluded" two from that first 
> page) that that's a bug.  
> 
> I think most people would agree that the "correct" way to exclude a 
> document would be to tie into the logic of executing the executing the 
> main query (like QEC does, or via a filter query) so that if a user asks 
> for the first 10 documents, you give them the first 10 documents - no 
> matter how many are being excluded.
> 
> 
> -Hoss

Re: Unable to build trunk

[Robert Muir]

Its not "wonky". you just have to ensure you have nothing else (like
some IDE, or build somewhere else) using ivy, then its safe to remove
the .lck file there.

I turned on this locking so that it hangs instead of causing cache
corruption, but ivy only has "simplelockfactory" so if you ^C at the
wrong time, it might leave a .lck file.

On Tue, Oct 30, 2012 at 9:27 AM, Erick Erickson  wrote:
> Not sure if it's relevant, but sometimes the ivy caches are wonky. Try
> deleting (on OS X) ~/.ivy2 recursively and building again? Of course
> your next build will download a bunch of jars...
>
> FWIW,
> Erick
>
> On Tue, Oct 30, 2012 at 5:38 AM, Markus Jelsma
>  wrote:
>> Hi,
>>
>> Since yesterday we're unable to build trunk and also a clean check out from 
>> trunk. We can compile the sources but not the example or dist.
>>
>> It hangs on resolve and after a while prints the following:
>>
>> resolve:
>>
>> [ivy:retrieve]
>> [ivy:retrieve] :: problems summary ::
>> [ivy:retrieve]  WARNINGS
>> [ivy:retrieve]  module not found: 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]   local: tried
>> [ivy:retrieve]
>> /home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
>> [ivy:retrieve]-- artifact 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
>> [ivy:retrieve]
>> /home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
>> [ivy:retrieve]   shared: tried
>> [ivy:retrieve]
>> /home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
>> [ivy:retrieve]-- artifact 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
>> [ivy:retrieve]
>> /home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
>> [ivy:retrieve]   public: tried
>> [ivy:retrieve]
>> http://repo1.maven.org/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
>> [ivy:retrieve]   sonatype-releases: tried
>> [ivy:retrieve]
>> http://oss.sonatype.org/content/repositories/releases/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
>> [ivy:retrieve]   working-chinese-mirror: tried
>> [ivy:retrieve]
>> http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
>> [ivy:retrieve]-- artifact 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
>> [ivy:retrieve]
>> http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.jar
>> [ivy:retrieve]  ::
>> [ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
>> [ivy:retrieve]  ::
>> [ivy:retrieve]  :: 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4: not found
>> [ivy:retrieve]  ::
>> [ivy:retrieve]  ERRORS
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]  impossible to acquire lock for 
>> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
>> [ivy:retrieve]
>> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>> BUILD FAILED
>> /home/markus/src/solr/trunk/solr/build.xml:336: The following error occurred 
>> while executing this line:
>> /home/markus/src/solr/trunk/solr/common-build.xml:345: The following error 
>> occurred while executing this line:
>> /home/markus/src/solr/trunk/solr/common-build.xml:388: The following error 
>> occurred while executing this line:
>> /home/markus/src/solr/trunk/lucene/common-build.xml:316: impossible to 
>> 

Re: Http 500/503 Responses from Solr

[Dmitry Kan]

cache tuning?
replication with load balancing?
Who knows!

(that is to say, that with this little info, one would only be guessing
about how to improve the capacity of your server).


On Tue, Oct 30, 2012 at 11:07 AM, deniz  wrote:

> Hi All,
>
> I ma running some load tests on my solr server, and as it could happen I am
> getting Http 500/503 responses randomly for some of my requests.. I know
> those responses are because server is too busy to handle that much of
> reqs...
>
> My queestion is...  how to increase the capacity of the server? What kind
> of
> settings do i need to change in in the solrconfig.xml ?
>
>
>
>
>
>
> -
> Zeki ama calismiyor... Calissa yapar...
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Http-500-503-Responses-from-Solr-tp4016948.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Regards,

Dmitry Kan

Re: Urgent Help Needed: Solr Data import problem

[Travis Low]

Like Amit said, this appears not to be a Solr problem. From the command
line of your machine, try this:

mysql -u'readonly' -p'readonly' -h'10.86.29.32' hpcms_db_new

If that works, and 10.86.29.32 is the server referenced by the URL in your
data-config.xml problem, then at least you know you have database
connectivity, and to the right server.

Also, if your unix server (presumably your mysql server) is 10.86.29.32,
then the URL in your data-config.xml is pointing to the wrong machine.  If
the one in the data-config.xml is correct, you need to test for
connectivity to that machine instead.

cheers,

Travis

On Tue, Oct 30, 2012 at 5:15 AM, kunal sachdeva wrote:

> Hi,
>
> This is my data-config file:-
>
> 
>
>   
>
>   
>
>  name="package" query="select concat('pckg', id) as id,pkg_name,updated_time
> from hp_package_info;">
> 
>
>  name="destination"
>  query="select name,id from hp_city">
>  
> 
> 
>   
> 
>
>
> and password is not null. and 10.86.29.32 is my unix server ip.
>
> regards,
> kunal
>
> On Tue, Oct 30, 2012 at 2:42 PM, Dave Stuart  wrote:
>
> > It looks as though you have a password set on your unix server. you will
> > need to either remove this or ti add the password into the connection
> string
> >
> > e.g. readonly:[yourpassword]@'10.86.29.32'
> >
> >
> >
> > >> 'readonly'@'10.86.29.32'
> > >> (using password: NO)"
> > On 30 Oct 2012, at 09:08, kunal sachdeva wrote:
> >
> > > Hi,
> > >
> > > I'm not getting this error while running in local machine. Please Help
> > >
> > > Regards,
> > > Kunal
> > >
> > > On Tue, Oct 30, 2012 at 10:32 AM, Amit Nithian 
> > wrote:
> > >
> > >> This looks like a MySQL permissions problem and not a Solr problem.
> > >> "Caused by: java.sql.SQLException: Access denied for user
> > >> 'readonly'@'10.86.29.32'
> > >> (using password: NO)"
> > >>
> > >> I'd advise reading your stack traces a bit more carefully. You should
> > >> check your permissions or if you don't own the DB, check with your DBA
> > >> to find out what user you should use to access your DB.
> > >>
> > >> - Amit
> > >>
> > >> On Mon, Oct 29, 2012 at 9:38 PM, kunal sachdeva
> > >>  wrote:
> > >>> Hi,
> > >>>
> > >>> I have tried using data-import in my local system. I was able to
> > execute
> > >> it
> > >>> properly. but when I tried to do it unix server I got following
> error:-
> > >>>
> > >>>
> > >>> INFO: Starting Full Import
> > >>> Oct 30, 2012 9:40:49 AM
> > >>> org.apache.solr.handler.dataimport.SimplePropertiesWriter
> > >>> readIndexerProperties
> > >>> WARNING: Unable to read: dataimport.properties
> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.update.DirectUpdateHandler2
> > >>> deleteAll
> > >>> INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
> onInit
> > >>> INFO: SolrDeletionPolicy.onInit: commits:num=1
> > >>>
> > >>>
> > >>
> >
> commit{dir=/opt/testsolr/multicore/core0/data/index,segFN=segments_1,version=1351490646879,generation=1,filenames=[segments_1]
> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
> > >>> updateCommits
> > >>> INFO: newest commit = 1351490646879
> > >>> Oct 30, 2012 9:40:49 AM
> > >> org.apache.solr.handler.dataimport.JdbcDataSource$1
> > >>> call
> > >>> INFO: Creating a connection for entity destination with URL:
> > >> jdbc:mysql://
> > >>> 172.16.37.160:3306/hpcms_db_new
> > >>> Oct 30, 2012 9:40:50 AM org.apache.solr.common.SolrException log
> > >>> SEVERE: Exception while processing: destination document :
> > >>>
> > >>
> >
> SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
> > >>> Unable to execute query: select name,id from hp_city Processing
> > Document
> > >> # 1
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)
> > >>> Caused by: java.lang.RuntimeException:
> > >>> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable
> > to
> > >>> execute query: select name,id from hp_city Processing Document # 1
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:621)
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)
> > >>>at
> > >>>
> > >>
> >
> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)
> > >>>... 3 more
> > >>> Caused by:
> > org.apache.solr.handler.dataimport.DataImportHandlerException:
> > >>> Unable to execute query: select name,id from hp_city Processing
> > Document
> > >> # 1
> > >>>at
> > >>>
> > >>
> >

SolrJ 4.0.0 addFilterQuery() issue ?

[Indika Tantrigoda]

Hi All,

I am using Solr 4.0 and SolrJ 4.0.0 to access Solr from a Spring web
application, and I seem to have an issue when querying Solr. If the
SolrQuery contains more than one addFilterQuery() I get an exception
stating,

org.apache.solr.common.SolrException:
org.apache.lucene.queryparser.classic.ParseException: Cannot parse
'[Ljava.lang.String;@1ec278b5': Encountered "" at line 1, column 28.
Was expecting one of:
"TO" ...
 ...
 ...

I am using the addFilterQuery() in the following manner:

solrQuery.addFilterQuery("{!field f=facet_state}CA");
solrQuery.addFilterQuery("{!field f=facet_city}Test City");

The issue only comes up if there are more than one addFilterQuery() or if
multiple parameters are passed to setFilterQueries() or addFilterQuery() -

solrQuery.setFilterQueries("{!field f=facet_state}CA", "{!field
f=facet_city}Test City");
solrQuery.addFilterQuery("{!field f=facet_state}CA", "{!field
f=facet_city}Test City");

Is there a new/alternate way in SolrJ 4 that this is done ?

Thanks in advance.
Indika

Solr question

[mkaran]

I have a question about integrating custom functionality into Solr.

What i want is the following scenario:
 1. A query is issued to Solr
 2. Solr uses my custom method to calculate the score (relevance) of each 
 document with respect to the query. 
 3. the results are returned sorted by descending score.

I am unsure how to implement this. 
>From the lucene documentation i thought it would be enough to implement
an extension of the Similarity class. Now im not so sure (i don't want to
just
override tf or idf i need a completely different scoring method, also i want
the
method to be applied to all documents and not only those that pass an
initial
boolean filter).

Which classes would i have to implement to get this behaviour?
Is this a good start?
http://lucene.apache.org/core/old_versioned_docs/versions/3_5_0/api/core/org/apache/lucene/search/package-summary.html#scoring)
 
Is there maybe a simpler way of achieving this?

Thanks for any advice,
Mladen




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-question-tp4017024.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Unable to build trunk

[Markus Jelsma]

Hi

I have no IDE but it likely i have terminated the build at some point, leaving 
a lock. Clearing the cache fixed the problem.

Thanks, 
 
-Original message-
> From:Robert Muir 
> Sent: Tue 30-Oct-2012 15:14
> To: solr-user@lucene.apache.org
> Subject: Re: Unable to build trunk
> 
> Its not "wonky". you just have to ensure you have nothing else (like
> some IDE, or build somewhere else) using ivy, then its safe to remove
> the .lck file there.
> 
> I turned on this locking so that it hangs instead of causing cache
> corruption, but ivy only has "simplelockfactory" so if you ^C at the
> wrong time, it might leave a .lck file.
> 
> On Tue, Oct 30, 2012 at 9:27 AM, Erick Erickson  
> wrote:
> > Not sure if it's relevant, but sometimes the ivy caches are wonky. Try
> > deleting (on OS X) ~/.ivy2 recursively and building again? Of course
> > your next build will download a bunch of jars...
> >
> > FWIW,
> > Erick
> >
> > On Tue, Oct 30, 2012 at 5:38 AM, Markus Jelsma
> >  wrote:
> >> Hi,
> >>
> >> Since yesterday we're unable to build trunk and also a clean check out 
> >> from trunk. We can compile the sources but not the example or dist.
> >>
> >> It hangs on resolve and after a while prints the following:
> >>
> >> resolve:
> >>
> >> [ivy:retrieve]
> >> [ivy:retrieve] :: problems summary ::
> >> [ivy:retrieve]  WARNINGS
> >> [ivy:retrieve]  module not found: 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]   local: tried
> >> [ivy:retrieve]
> >> /home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
> >> [ivy:retrieve]-- artifact 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
> >> [ivy:retrieve]
> >> /home/markus/.ivy2/local/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
> >> [ivy:retrieve]   shared: tried
> >> [ivy:retrieve]
> >> /home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/ivys/ivy.xml
> >> [ivy:retrieve]-- artifact 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
> >> [ivy:retrieve]
> >> /home/markus/.ivy2/shared/com.carrotsearch.randomizedtesting/randomizedtesting-runner/2.0.4/jars/randomizedtesting-runner.jar
> >> [ivy:retrieve]   public: tried
> >> [ivy:retrieve]
> >> http://repo1.maven.org/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
> >> [ivy:retrieve]   sonatype-releases: tried
> >> [ivy:retrieve]
> >> http://oss.sonatype.org/content/repositories/releases/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
> >> [ivy:retrieve]   working-chinese-mirror: tried
> >> [ivy:retrieve]
> >> http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.pom
> >> [ivy:retrieve]-- artifact 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4!randomizedtesting-runner.jar:
> >> [ivy:retrieve]
> >> http://mirror.netcologne.de/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.0.4/randomizedtesting-runner-2.0.4.jar
> >> [ivy:retrieve]  ::
> >> [ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
> >> [ivy:retrieve]  ::
> >> [ivy:retrieve]  :: 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4: not 
> >> found
> >> [ivy:retrieve]  ::
> >> [ivy:retrieve]  ERRORS
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]  impossible to acquire lock for 
> >> com.carrotsearch.randomizedtesting#randomizedtesting-runner;2.0.4
> >> [ivy:retrieve]
> >> [ivy:retrieve] :: USE VERBOSE OR DEBUG M

Re: Items disappearing from Solr index

[Kissue Kissue]

I have encountered another case where deleteByQuery fails. It fails for
when i have a catalogueId value "www"
and thus issue the query ( {!term
f=catalogueId}www). One of my customers
just reported this now. Any ideas why a value like that when issued in a
deleteByQuery should be wiping out the entire index?

Thanks.

On Thu, Sep 27, 2012 at 2:27 PM, Kissue Kissue  wrote:

> Actually this problem occurs even when i am doing just deletes. I tested
> by sending only one delete query for a single catalogue and had the same
> problem. I always optimize once.
>
> I changed to the syntax you suggested ( {!term f=catalogueId}Emory Labs)
> and works like a charm. Thanks for the pointer, saved me from another issue
> that could have occurred at some point.
>
> Thanks.
>
>
>
>
> On Thu, Sep 27, 2012 at 12:30 PM, Erick Erickson 
> wrote:
>
>> Wild shot in the dark
>>
>> What happens if you switch from StreamingUpdateSolrServer to
>> HttpSolrServer?
>>
>> What I'm wondering is if somehow you're getting a queueing problem. If
>> you have
>> multiple threads defined for SUSS, it might be possible (and I'm
>> guessing) that
>> the delete bit is getting sent after some of the adds. Frankly I doubt
>> this is
>> the case, but this issue is so weird that I'm grasping at straws.
>>
>> BTW, there's no reason to optimize twice. Actually, the new thinking is
>> that
>> optimizing usually isn't necessary anyway. But if you insist on optimizing
>> there's no reason to do it _both_ after the deletes and after the adds,
>> just
>> do it after the adds.
>>
>> Best
>> Erick
>>
>> On Thu, Sep 27, 2012 at 4:31 AM, Kissue Kissue 
>> wrote:
>> > #What is the field type for that field - string or text?
>> >
>> > It is a string type.
>> >
>> > Thanks.
>> >
>> > On Wed, Sep 26, 2012 at 8:14 PM, Jack Krupansky <
>> j...@basetechnology.com>wrote:
>> >
>> >> What is the field type for that field - string or text?
>> >>
>> >>
>> >> -- Jack Krupansky
>> >>
>> >> -Original Message- From: Kissue Kissue
>> >> Sent: Wednesday, September 26, 2012 1:43 PM
>> >>
>> >> To: solr-user@lucene.apache.org
>> >> Subject: Re: Items disappearing from Solr index
>> >>
>> >> # It is looking for documents with "Emory" in the specified field OR
>> "Labs"
>> >> in the default search field.
>> >>
>> >> This does not seem to be the case. For instance issuing a
>> deleteByQuery for
>> >> catalogueId: "PEARL LINGUISTICS LTD" also deletes the contents of a
>> >> catalogueId with the value: "Ncl_**MacNaughtonMcGregorCoaching_**
>> >> vf010811".
>> >>
>> >> Thanks.
>> >>
>> >> On Wed, Sep 26, 2012 at 2:37 PM, Jack Krupansky <
>> j...@basetechnology.com>*
>> >> *wrote:
>> >>
>> >>  It is looking for documents with "Emory" in the specified field OR
>> "Labs"
>> >>> in the default search field.
>> >>>
>> >>> -- Jack Krupansky
>> >>>
>> >>> -Original Message- From: Kissue Kissue
>> >>> Sent: Wednesday, September 26, 2012 7:47 AM
>> >>> To: solr-user@lucene.apache.org
>> >>> Subject: Re: Items disappearing from Solr index
>> >>>
>> >>>
>> >>> I have just solved this problem.
>> >>>
>> >>> We have a field called catalogueId. One possible value for this field
>> >>> could
>> >>> be "Emory Labs". I found out that when the following delete by query
>> is
>> >>> sent to solr:
>> >>>
>> >>> getSolrServer().deleteByQuery(catalogueId + ":" + Emory Labs)
>> >>>  [Notice
>> >>>
>> >>> that
>> >>> there are no quotes surrounding the catalogueId value - Emory Labs]
>> >>>
>> >>> For some reason this delete by query ends up deleting the contents of
>> some
>> >>> other random catalogues too which is the reason why we are loosing
>> items
>> >>> from the index. When the query is changed to:
>> >>>
>> >>> getSolrServer().deleteByQuery(catalogueId + ":" + "Emory Labs"),
>> >>> then it
>> >>>
>> >>> starts to correctly delete only items in the Emory Labs catalogue.
>> >>>
>> >>> So my first question is, what exactly does deleteByQuery do in the
>> first
>> >>> query without the quotes? How is it determining which catalogues to
>> >>> delete?
>> >>>
>> >>> Secondly, shouldn't the correct behaviour be not to delete anything
>> at all
>> >>> in this case since when a search is done for the same catalogueId
>> without
>> >>> the quotes it just simply returns no results?
>> >>>
>> >>> Thanks.
>> >>>
>> >>>
>> >>> On Mon, Sep 24, 2012 at 3:12 PM, Kissue Kissue 
>> >>> wrote:
>> >>>
>> >>>  Hi Erick,
>> >>>
>> 
>>  Thanks for your reply. Yes i am using delete by query. I am currently
>>  logging the number of items to be deleted before handing off to
>> solr. And
>>  from solr logs i can it deleted exactly that number. I will verify
>>  further.
>> 
>>  Thanks.
>> 
>> 
>>  On Mon, Sep 24, 2012 at 1:21 PM, Erick Erickson <
>> erickerick...@gmail.com
>>  >
>>  **wrote:
>> 
>> 
>>   How do you delete items? By ID or by query?
>> 
>> >
>> >

solr indexing using jquery AJAX

[Amit Mallik]

Hi I am trying to index using AJAX basically jquery.
Below is my code
try {
$.ajax({
type: "POST",
url: "http://myserver:8080/solr/update?commit=true";,
data: "20name=name>trailblazers",

contentType: "text/xml",
success: function (data) { alert(data); },
failure: function (errMsg) {
alert(errMsg);
}
});
}

I see the request in server as 400. Not sure what is wrong.
192.168.11.88 - - [30/Oct/2012:19:31:54 +0530] "OPTIONS 
/solr/update?commit=true HTTP/1.1" 400 1052


My question is
1. Is it possible to index solr using jquery, ajax.
2. If so what's wrong in the above format.

Re: solr indexing using jquery AJAX

[Gora Mohanty]

On 30 October 2012 20:16, Amit Mallik  wrote:
> Hi I am trying to index using AJAX basically jquery.
> Below is my code
> try {
> $.ajax({
> type: "POST",
> url:
> "http://myserver:8080/solr/update?commit=true";,
>
> data: "20 name=name>trailblazers",
[...]

Am away from where I can check this, but your  Solr XMl seems ill-formed.
Please take a look at example/exampledocs/mem.xml in a Solr source
directory for an example. The "name" attribute for each field needs to be
quoted, and here the quotes would need to be escaped. Please see
http://stackoverflow.com/questions/3099369/jquery-ajax-post-to-web-service
for an example of a proper XML post from jQuery.ajax

Regards,
Gora

Re: solr indexing using jquery AJAX

[Michael Della Bitta]

Also I believe you have to set the MIME type on your post.

You might want to refer to the server side logs to see what Solr is
saying about your request. There likely will be a descriptive error
there.

Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Oct 30, 2012 at 11:44 AM, Gora Mohanty  wrote:
> On 30 October 2012 20:16, Amit Mallik  wrote:
>> Hi I am trying to index using AJAX basically jquery.
>> Below is my code
>> try {
>> $.ajax({
>> type: "POST",
>> url:
>> "http://myserver:8080/solr/update?commit=true";,
>>
>> data: "20> name=name>trailblazers",
> [...]
>
> Am away from where I can check this, but your  Solr XMl seems ill-formed.
> Please take a look at example/exampledocs/mem.xml in a Solr source
> directory for an example. The "name" attribute for each field needs to be
> quoted, and here the quotes would need to be escaped. Please see
> http://stackoverflow.com/questions/3099369/jquery-ajax-post-to-web-service
> for an example of a proper XML post from jQuery.ajax
>
> Regards,
> Gora

SolrCloud AutoSharding? In enterprise environment?

[joseph_12345]

Hi All,

I have looked at the below post URL and it really helps me a lot.

http://lucene.472066.n3.nabble.com/SolrCloud-AutoSharding-td4011834.html 

But I have few questions. 

We are exploring SolrCloud to index millions of product details. We are
planning to use a complete pool for SollCloud with a set of 15 physical
machines. The clients will be hitting the VIP URL of SolrCloud pool instead
of individual machine names or IP addresses. So my questions are, 

1. While indexing the product details, do I need to take care of custom
sharding strategy or I just need to specify the number of Shards as 15(total
number of boxes in pool) and the SolrCloud takes care of sharding
internally? 

2. If SolrCloud takes care of sharding since this shards are in 15 different
boxes, does SolrCloud internally do full scan to all these boxes if client
is querying by specifying the VIP url (Not the individual machine names)? Or
does SolrCloud do some extra intelligent logic to avoid full scan? 

3. I couldn't find proper documentation about what SolrCloud does internally
in terms of Sharding and what developer has to do to optimize queries? 

Any response on this is greatly appreciated. 

Thanks 
Joseph



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-AutoSharding-In-enterprise-environment-tp4017036.html
Sent from the Solr - User mailing list archive at Nabble.com.

Grouping based on multiple criteria

[Alan Woodward]

Hi list,

I'd like to be able to present a list of results which are grouped on a single 
field, but then show various members of each group according to several 
different criteria.  So for example, for e-commerce search, we group at the top 
level by the vendor, but then show the most expensive item, least expensive 
item, most heavily discounted item, etc.

I can't find anything that would let me do this in the current grouping code.  
I'm thinking I'd need to implement a form of TopFieldCollector that maintained 
multiple sort orders that could be used for the second pass collector, but 
there doesn't seem to be anywhere to plug that in easily.

Is there anything already out there that I'm missing, or do I have to do some 
actual work?  :-)

Thanks, Alan

RE: Nodes cannot recover and become unavailable

[balaji.gandhi]

Hi Team,

What is the preferred way to upgrade from SOLR 4.0.0-BETA to SOLR 4.0.0?

We saw the same errors happening when we did the upgrade:-

Oct 29, 2012 4:55:00 PM org.apache.solr.common.SolrException log
SEVERE: Error while trying to recover.
core=mediacms:org.apache.solr.common.SolrException: We are not the leader
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:401)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at
org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:199)
at
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:388)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:220)

Thanks,
Balaji



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Nodes-cannot-recover-and-become-unavailable-tp4008916p4017037.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: [Announce] Apache Solr 4.0 with RankingAlgorithm 1.4.4 and Realtime NRT available for download

[Michael Della Bitta]

Happy to help!

Cheers,

Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Mon, Oct 29, 2012 at 11:37 PM, Nagendra Nagarajayya
 wrote:
> Thanks Michael for the feedback. Will take a look at this ...
>
>
> Regards,
>
> Nagendra Nagarajayya
> http://solr-ra.tgels.org
> http://rankingalgorithm.tgels.org
>
>
> On 10/29/2012 9:17 AM, Michael Della Bitta wrote:
>>
>> As an external observer, I think the main problem is your branding.
>> "Realtime Near Realtime" is definitely an oxymoron, and your ranking
>> algorithm is called "Ranking Algorithm," which is generic enough to
>> suggest that a. it's the only ranking algorithm available, and b. by
>> implication, that Solr doesn't have one built in.
>>
>> I would suggest two improvements:
>>
>> 1. Come up with a top-level name for your overall efforts. Apache
>> Foundation has 'Apache,' so automatic branding of every component they
>> build. Then your ranking algorithm could be called "Tgels Ranking
>> Algorithm for Apache Solr" (for example), which is totally legit. And
>> "Tgels Realtime Search for Apache Solr."
>>
>> 2. Maybe point out that you're building on top of the work of the
>> Apache Solr and Lucene projects a little more prominently.
>>
>> I think with those two little tweaks, you'd actually very easily get
>> more people interested in your contributions.
>>
>> Just my two cents,
>>
>> Michael Della Bitta
>>
>> 
>> Appinions
>> 18 East 41st Street, 2nd Floor
>> New York, NY 10017-6271
>>
>> www.appinions.com
>>
>> Where Influence Isn’t a Game
>>
>>
>> On Mon, Oct 29, 2012 at 11:35 AM, Nagendra Nagarajayya
>>  wrote:
>>>
>>> Jack:
>>>
>>> I respect your hard-work responding to user problems on the mail list. So
>>> it
>>> would be nicer to try out Realtime NRT then pass rogue comments, whether
>>> a
>>> contribution is legit/spam or a scam... I guess it illuminates the narrow
>>> minded view of oneself ...  The spirit of open source is contributions
>>> from
>>> not only commiters but other developers, from the Solr wiki "A half-baked
>>> patch in Jira, with no documentation, no tests and no backwards
>>> compatibility is better than no patch at all."
>>>
>>> You would gain more respect if you actually download realtime-nrt, check
>>> out
>>> if it does provide a view of a realtime index compared to a
>>> point-in-time
>>> snapshot, see if you can understand the code and provide clarity  and
>>> feedback to the list if you do find problems with it. realtime-nrt offers
>>> search capability as to realtime-get. Checkout if  this is true ... I
>>> would
>>> really welcome your comments on the list or through the JIRA here:
>>>
>>> https://issues.apache.org/jira/browse/SOLR-3816
>>>
>>>
>>> Regards,
>>>
>>> Nagendra Nagarajayya
>>> http://solr-ra.tgels.org
>>> http://rankingalgorithm.tgels.org
>>>
>>> On 10/29/2012 7:30 AM, Jack Krupansky wrote:

 Could any of the committers here confirm whether this is a legitimate
 effort? I mean, how could anything labeled "Apache ABC with XYZ" be an
 "external project" and be sanctioned/licensed by Apache? In fact, the
 linked
 web page doesn't even acknowledge the ownership of the Apache trademarks
 or
 ASL. And the term "Realtime NRT" is nonsensical. Even worse: "Realtime
 NRT
 makes available a near realtime view". Equally nonsensical. Who knows,
 maybe
 it is legit, but it sure comes across as a scam/spam.

 -- Jack Krupansky

 -Original Message- From: Nagendra Nagarajayya
 Sent: Monday, October 29, 2012 10:06 AM
 To: solr-user@lucene.apache.org
 Subject: [Announce] Apache Solr 4.0 with RankingAlgorithm 1.4.4 and
 Realtime NRT available for download

 Hi!

 I am very excited to announce the availability of Apache Solr 4.0 with
 RankingAlgorithm 1.4.4 and Realtime NRT. Realtime NRT is a high
 performance and more granular NRT implementation as to soft commit. The
 update performance is about 70,000 documents / sec* (almost 1.5-2x
 performance improvement over soft-commit). You can also scale up to 2
 billion documents* in a single core, and query half a billion documents
 index in ms**. Realtime NRT is different from realtime-get. realtime-get
 does not have search capability and is a lookup by id. Realtime NRT
 allows full search, see here 
 for more info.

 Realtime NRT has been contributed back to Solr, see JIRA:
 https://issues.apache.org/jira/browse/SOLR-3816

 RankingAlgorithm 1.4.4 supports the entire Lucene Query Syntax, ą and/or
 boolean/dismax/boost queries and is compatible with the new Lucene 4.0
 api.

 You can get more information about Solr 4.0 with RankingAlgorithm 1.4.4
 and Realtime NRT performance

trunk is unable to replicate between nodes ( Unable to download ... completely)

[Markus Jelsma]

Hi,

We're testing again with today's trunk and using the new Lucene 4.1 format by 
default. When nodes are not restarted things are kind of stable but restarting 
nodes leads to a lot of mayhem. It seems we can get the cluster back up and 
running by clearing ZK and restarting everything (another issue) but 
replication becomes impossible for some nodes leading to a continuous state of 
failing recovery etc.

Here are some excepts from the logs:

2012-10-30 16:12:39,674 ERROR [solr.servlet.SolrDispatchFilter] - [http-8080-exe
c-5] - : null:java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkBounds(Buffer.java:530)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:218)
at org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferInde
xInput.java:91)
at org.apache.solr.handler.ReplicationHandler$DirectoryFileStream.write(
ReplicationHandler.java:1065)
at 
org.apache.solr.handler.ReplicationHandler$3.write(ReplicationHandler.java:932)


2012-10-30 16:10:32,220 ERROR [solr.handler.ReplicationHandler] - [RecoveryThrea
d] - : SnapPull failed :org.apache.solr.common.SolrException: Unable to download
 _x.fdt completely. Downloaded 13631488!=13843504
at org.apache.solr.handler.SnapPuller$DirectoryFileFetcher.cleanup(SnapP
uller.java:1237)
at org.apache.solr.handler.SnapPuller$DirectoryFileFetcher.fetchFile(Sna
pPuller.java:1118)
at org.apache.solr.handler.SnapPuller.downloadIndexFiles(SnapPuller.java
:716)
at 
org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:387)
at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:273)
at 
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:152)
at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:407)

2012-10-30 16:12:51,061 WARN [solr.handler.ReplicationHandler] - [http-8080-exec
-3] - : Exception while writing response for params: file=_p_Lucene41_0.doc&comm
and=filecontent&checksum=true&generation=6&qt=/replication&wt=filestream
java.io.EOFException: read past EOF: 
MMapIndexInput(path="/opt/solr/cores/openindex_h/data/index.20121030152234973/_p_Lucene41_0.doc")
at 
org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferIndexInput.java:100)
at 
org.apache.solr.handler.ReplicationHandler$DirectoryFileStream.write(ReplicationHandler.java:1065)
at 
org.apache.solr.handler.ReplicationHandler$3.write(ReplicationHandler.java:932)


Needless to say i'm puzzled so i'm wondering if anyone has seen this before or 
have some hints that might help digg further.

Thanks,
Markus

RE: trunk is unable to replicate between nodes ( Unable to download ... completely)

[Markus Jelsma]

Ah, we're also seeing Solr lookup an unexisting directory:

2012-10-30 16:32:26,578 ERROR [handler.admin.CoreAdminHandler] - 
[http-8080-exec-2] - : IO error while trying to get the size of the 
Directory:org.apache.lucene.store.NoSuchDirectoryException: directory 
'/opt/solr/cores/shard_a/data/index' does not exist
at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:220)
at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:243)
at 
org.apache.lucene.store.NRTCachingDirectory.listAll(NRTCachingDirectory.java:132)
at 
org.apache.solr.core.DirectoryFactory.sizeOfDirectory(DirectoryFactory.java:146)

Instead of data/index it should be looking for data/index.20121030152324761/, 
which actually does exist.

 
 
-Original message-
> From:Markus Jelsma 
> Sent: Tue 30-Oct-2012 17:30
> To: solr-user@lucene.apache.org
> Subject: trunk is unable to replicate between nodes ( Unable to download ... 
> completely)
> 
> Hi,
> 
> We're testing again with today's trunk and using the new Lucene 4.1 format by 
> default. When nodes are not restarted things are kind of stable but 
> restarting nodes leads to a lot of mayhem. It seems we can get the cluster 
> back up and running by clearing ZK and restarting everything (another issue) 
> but replication becomes impossible for some nodes leading to a continuous 
> state of failing recovery etc.
> 
> Here are some excepts from the logs:
> 
> 2012-10-30 16:12:39,674 ERROR [solr.servlet.SolrDispatchFilter] - 
> [http-8080-exe
> c-5] - : null:java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkBounds(Buffer.java:530)
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:218)
> at 
> org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferInde
> xInput.java:91)
> at 
> org.apache.solr.handler.ReplicationHandler$DirectoryFileStream.write(
> ReplicationHandler.java:1065)
> at 
> org.apache.solr.handler.ReplicationHandler$3.write(ReplicationHandler.java:932)
> 
> 
> 2012-10-30 16:10:32,220 ERROR [solr.handler.ReplicationHandler] - 
> [RecoveryThrea
> d] - : SnapPull failed :org.apache.solr.common.SolrException: Unable to 
> download
>  _x.fdt completely. Downloaded 13631488!=13843504
> at 
> org.apache.solr.handler.SnapPuller$DirectoryFileFetcher.cleanup(SnapP
> uller.java:1237)
> at 
> org.apache.solr.handler.SnapPuller$DirectoryFileFetcher.fetchFile(Sna
> pPuller.java:1118)
> at 
> org.apache.solr.handler.SnapPuller.downloadIndexFiles(SnapPuller.java
> :716)
> at 
> org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:387)
> at 
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:273)
> at 
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:152)
> at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:407)
> 
> 2012-10-30 16:12:51,061 WARN [solr.handler.ReplicationHandler] - 
> [http-8080-exec
> -3] - : Exception while writing response for params: 
> file=_p_Lucene41_0.doc&comm
> and=filecontent&checksum=true&generation=6&qt=/replication&wt=filestream
> java.io.EOFException: read past EOF: 
> MMapIndexInput(path="/opt/solr/cores/openindex_h/data/index.20121030152234973/_p_Lucene41_0.doc")
> at 
> org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferIndexInput.java:100)
> at 
> org.apache.solr.handler.ReplicationHandler$DirectoryFileStream.write(ReplicationHandler.java:1065)
> at 
> org.apache.solr.handler.ReplicationHandler$3.write(ReplicationHandler.java:932)
> 
> 
> Needless to say i'm puzzled so i'm wondering if anyone has seen this before 
> or have some hints that might help digg further.
> 
> Thanks,
> Markus
> 

Forwardslash delimiter.Solr4.0 query for path like /Customer/Content/*

[Dirk Högemann]

Hi,

I am currently upgrading from Solr 3.5 to Solr 4.0

I used to have filter-bases restrictions for my search based on the paths
of documents in a content repository.
E.g.  fq={!q.op=OR df=}folderPath_}/customer/content/*

Unfortunately this does not work anymore, as lucene now supports
Regexpsearches - delimiting the expression with forward slashes:
http://lucene.apache.org/core/4_0_0-BETA/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Regexp_Searches

this leads to a parsed query, which is of course not what is intended:

RegexpQuery(folderPath_:/standardlsg/)
folderPath_:shareddocs RegexpQuery(folderPath_:/personen/)
folderPath_:*

Is there a possibility to make the example query above work, without
escaping the "/" with "\/"?
Otherwise I will have to parse all queries  (coming from persisted
configurations in the repositiory) and escape the relevant parts of the
queries on that field, which is somewhat ugly...

The field I search on is of type:




  
  


 

Best and thanks for any hints
Dirk

Re: how to display MathML in search results?

[Dave Stuart]

Hi Joe,

I have a suspect that this is a drupal thing as opposed to Solr specifically. 
Probably the best approach would be to add a extra field that is the raw data 
for display instead of using the content field which get a fair amount of 
processing.

I would recommend opening a issue on the apachesolr module issues queue and ill 
answer it there as I dont want to spam the solr crowd with drupal related stuff

Regards,

Dave



On 30 Oct 2012, at 16:56, Joe Corneli wrote:

> Dear Solr experts:
> 
> I'm running Solr under Drupal 7 on a site with a lot of mathematics
> written in MathML.
> 
> Search results are returned with the MathML formatting removed -- in
> other words, the system attempts to show the results as if they were
> plain text.
> 
> You can see that in action here:
> 
> http://beta.planetmath.org/search/site/formula
> 
> Or, in the underlying XML:
> 
> http://beta.planetmath.org:8983/solr/select?indent=on&version=2.2&q=formula&fq=&start=0&rows=10&fl=*%2Cscore&wt=&explainOther=&hl.fl=
> 
> BTW: I apologize for some ridiculous and unrelated encoding errors
> that I hope don't confuse the point here - still trying to sort these
> out.  The basic point is that you shouldn't see things like "an
> additive function f f f", but rather, the rendered version of
> 
> an additive function  display="inline" xref="I1.i1.p1.1.m1.1.cmml"> id="I1.i1.p1.1.m1.1a" xref="I1.i1.p1.1.m1.1.cmml"> id="I1.i1.p1.1.m1.1.1"
> xref="I1.i1.p1.1.m1.1.1.cmml">f id="I1.i1.p1.1.m1.1.cmml" encoding="MathML-Content"
> xref="I1.i1.p1.1.m1.1"> xref="I1.i1.p1.1.m1.1.1">f id="I1.i1.p1.1.m1.1b" encoding="application/x-tex"
> xref="I1.i1.p1.1.m1.1.cmml">f
> 
> Here just "an additive function f" with "f" in italics.
> 
> Please advise!
> Thanks,
> Joe

David Stuart
M  +44(0) 778 854 2157
T   +44(0) 845 519 5465
www.axistwelve.com
Axis12 Ltd | 7 Wynford Road
| London | N1 9QN | UK

AXIS12 - Enterprise Web Solutions

Reg Company No. 7215135
VAT No. 997 4801 60

This e-mail is strictly confidential and intended solely for the ordinary user 
of the e-mail account to which it is addressed. If you have received this 
e-mail in error please inform Axis12 immediately by return e-mail or telephone. 
We advise that in keeping with good computing practice the recipient of this 
e-mail should ensure that it is virus free. We do not accept any responsibility 
for any loss or damage that may arise from the use of this email or its 
contents.

Re: how to display MathML in search results?

[Joe Corneli]

Hi Dave,

Thanks for your offer to help!

I moved the original post to a support request here:
http://drupal.org/node/1827260

(Noting: display of the XHTML works fine in nodes themselves...)

Joe

On Tue, Oct 30, 2012 at 5:23 PM, Dave Stuart  wrote:
> Hi Joe,
>
> I have a suspect that this is a drupal thing as opposed to Solr specifically. 
> Probably the best approach would be to add a extra field that is the raw data 
> for display instead of using the content field which get a fair amount of 
> processing.
>
> I would recommend opening a issue on the apachesolr module issues queue and 
> ill answer it there as I dont want to spam the solr crowd with drupal related 
> stuff
>
> Regards,
>
> Dave
>
>
>
> On 30 Oct 2012, at 16:56, Joe Corneli wrote:
>
>> Dear Solr experts:
>>
>> I'm running Solr under Drupal 7 on a site with a lot of mathematics
>> written in MathML.
>>
>> Search results are returned with the MathML formatting removed -- in
>> other words, the system attempts to show the results as if they were
>> plain text.
>>
>> You can see that in action here:
>>
>> http://beta.planetmath.org/search/site/formula
>>
>> Or, in the underlying XML:
>>
>> http://beta.planetmath.org:8983/solr/select?indent=on&version=2.2&q=formula&fq=&start=0&rows=10&fl=*%2Cscore&wt=&explainOther=&hl.fl=
>>
>> BTW: I apologize for some ridiculous and unrelated encoding errors
>> that I hope don't confuse the point here - still trying to sort these
>> out.  The basic point is that you shouldn't see things like "an
>> additive function f f f", but rather, the rendered version of
>>
>> an additive function > display="inline" xref="I1.i1.p1.1.m1.1.cmml">> id="I1.i1.p1.1.m1.1a" xref="I1.i1.p1.1.m1.1.cmml">> id="I1.i1.p1.1.m1.1.1"
>> xref="I1.i1.p1.1.m1.1.1.cmml">f> id="I1.i1.p1.1.m1.1.cmml" encoding="MathML-Content"
>> xref="I1.i1.p1.1.m1.1">> xref="I1.i1.p1.1.m1.1.1">f> id="I1.i1.p1.1.m1.1b" encoding="application/x-tex"
>> xref="I1.i1.p1.1.m1.1.cmml">f
>>
>> Here just "an additive function f" with "f" in italics.
>>
>> Please advise!
>> Thanks,
>> Joe
>
> David Stuart
> M  +44(0) 778 854 2157
> T   +44(0) 845 519 5465
> www.axistwelve.com
> Axis12 Ltd | 7 Wynford Road
> | London | N1 9QN | UK
>
> AXIS12 - Enterprise Web Solutions
>
> Reg Company No. 7215135
> VAT No. 997 4801 60
>
> This e-mail is strictly confidential and intended solely for the ordinary 
> user of the e-mail account to which it is addressed. If you have received 
> this e-mail in error please inform Axis12 immediately by return e-mail or 
> telephone. We advise that in keeping with good computing practice the 
> recipient of this e-mail should ensure that it is virus free. We do not 
> accept any responsibility for any loss or damage that may arise from the use 
> of this email or its contents.
>
>
>
>

Re: how to display MathML in search results?

[Paul Libbrecht]

Joe,

if XHTML works fine... why would MathML not?
Is it swallowed?
I agree with Dave that I see nothing Solr specific.
Maybe a namespace issue?

If the search results pull from Solr, they would pull from a stored field which 
you can inspect by using the url /solr/select?q= (this renders XML, see the 
Solr tutorial). You'd find a clue when it gets dropped.

Paul


Le 30 oct. 2012 à 18:47, Joe Corneli a écrit :

> Hi Dave,
> 
> Thanks for your offer to help!
> 
> I moved the original post to a support request here:
> http://drupal.org/node/1827260
> 
> (Noting: display of the XHTML works fine in nodes themselves...)
> 
> Joe
> 
> On Tue, Oct 30, 2012 at 5:23 PM, Dave Stuart  wrote:
>> Hi Joe,
>> 
>> I have a suspect that this is a drupal thing as opposed to Solr 
>> specifically. Probably the best approach would be to add a extra field that 
>> is the raw data for display instead of using the content field which get a 
>> fair amount of processing.
>> 
>> I would recommend opening a issue on the apachesolr module issues queue and 
>> ill answer it there as I dont want to spam the solr crowd with drupal 
>> related stuff
>> 
>> Regards,
>> 
>> Dave
>> 
>> 
>> 
>> On 30 Oct 2012, at 16:56, Joe Corneli wrote:
>> 
>>> Dear Solr experts:
>>> 
>>> I'm running Solr under Drupal 7 on a site with a lot of mathematics
>>> written in MathML.
>>> 
>>> Search results are returned with the MathML formatting removed -- in
>>> other words, the system attempts to show the results as if they were
>>> plain text.
>>> 
>>> You can see that in action here:
>>> 
>>> http://beta.planetmath.org/search/site/formula
>>> 
>>> Or, in the underlying XML:
>>> 
>>> http://beta.planetmath.org:8983/solr/select?indent=on&version=2.2&q=formula&fq=&start=0&rows=10&fl=*%2Cscore&wt=&explainOther=&hl.fl=
>>> 
>>> BTW: I apologize for some ridiculous and unrelated encoding errors
>>> that I hope don't confuse the point here - still trying to sort these
>>> out.  The basic point is that you shouldn't see things like "an
>>> additive function f f f", but rather, the rendered version of
>>> 
>>> an additive function >> display="inline" xref="I1.i1.p1.1.m1.1.cmml">>> id="I1.i1.p1.1.m1.1a" xref="I1.i1.p1.1.m1.1.cmml">>> id="I1.i1.p1.1.m1.1.1"
>>> xref="I1.i1.p1.1.m1.1.1.cmml">f>> id="I1.i1.p1.1.m1.1.cmml" encoding="MathML-Content"
>>> xref="I1.i1.p1.1.m1.1">>> xref="I1.i1.p1.1.m1.1.1">f>> id="I1.i1.p1.1.m1.1b" encoding="application/x-tex"
>>> xref="I1.i1.p1.1.m1.1.cmml">f
>>> 
>>> Here just "an additive function f" with "f" in italics.
>>> 
>>> Please advise!
>>> Thanks,
>>> Joe
>> 
>> David Stuart
>> M  +44(0) 778 854 2157
>> T   +44(0) 845 519 5465
>> www.axistwelve.com
>> Axis12 Ltd | 7 Wynford Road
>> | London | N1 9QN | UK
>> 
>> AXIS12 - Enterprise Web Solutions
>> 
>> Reg Company No. 7215135
>> VAT No. 997 4801 60
>> 
>> This e-mail is strictly confidential and intended solely for the ordinary 
>> user of the e-mail account to which it is addressed. If you have received 
>> this e-mail in error please inform Axis12 immediately by return e-mail or 
>> telephone. We advise that in keeping with good computing practice the 
>> recipient of this e-mail should ensure that it is virus free. We do not 
>> accept any responsibility for any loss or damage that may arise from the use 
>> of this email or its contents.
>> 
>> 
>> 
>> 

Re: Subqueries... using results from the main query

[Michael Della Bitta]

Not to my knowledge. The closest thing is joins I think. My company
hasn't had a need for features like this, so I'm not 100% sure.

You're probably going to have to be a bit more patient to get a
canonical answer.

Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Oct 30, 2012 at 1:58 PM, Gustav  wrote:
> Anyone?
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Subqueries-using-results-from-the-main-query-tp4017000p4017084.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Urgent Help Needed: Solr Data import problem

[Amit Nithian]

This error is typically because of a mysql permissions problem. These
are usually resolved by a GRANT statement on your DB to allow for
users to connect remotely to your database server.

I don't know the full syntax but a quick search on Google should yield
what you are looking for. If you don't control access to this DB, talk
to your sys admin who does maintain this access and s/he should be
able to help resolve this.

On Tue, Oct 30, 2012 at 7:13 AM, Travis Low  wrote:
> Like Amit said, this appears not to be a Solr problem. From the command
> line of your machine, try this:
>
> mysql -u'readonly' -p'readonly' -h'10.86.29.32' hpcms_db_new
>
> If that works, and 10.86.29.32 is the server referenced by the URL in your
> data-config.xml problem, then at least you know you have database
> connectivity, and to the right server.
>
> Also, if your unix server (presumably your mysql server) is 10.86.29.32,
> then the URL in your data-config.xml is pointing to the wrong machine.  If
> the one in the data-config.xml is correct, you need to test for
> connectivity to that machine instead.
>
> cheers,
>
> Travis
>
> On Tue, Oct 30, 2012 at 5:15 AM, kunal sachdeva 
> wrote:
>
>> Hi,
>>
>> This is my data-config file:-
>>
>> 
>>
>>   
>>
>>   
>>
>> > name="package" query="select concat('pckg', id) as id,pkg_name,updated_time
>> from hp_package_info;">
>> 
>>
>> > name="destination"
>>  query="select name,id from hp_city">
>>  
>> 
>> 
>>   
>> 
>>
>>
>> and password is not null. and 10.86.29.32 is my unix server ip.
>>
>> regards,
>> kunal
>>
>> On Tue, Oct 30, 2012 at 2:42 PM, Dave Stuart  wrote:
>>
>> > It looks as though you have a password set on your unix server. you will
>> > need to either remove this or ti add the password into the connection
>> string
>> >
>> > e.g. readonly:[yourpassword]@'10.86.29.32'
>> >
>> >
>> >
>> > >> 'readonly'@'10.86.29.32'
>> > >> (using password: NO)"
>> > On 30 Oct 2012, at 09:08, kunal sachdeva wrote:
>> >
>> > > Hi,
>> > >
>> > > I'm not getting this error while running in local machine. Please Help
>> > >
>> > > Regards,
>> > > Kunal
>> > >
>> > > On Tue, Oct 30, 2012 at 10:32 AM, Amit Nithian 
>> > wrote:
>> > >
>> > >> This looks like a MySQL permissions problem and not a Solr problem.
>> > >> "Caused by: java.sql.SQLException: Access denied for user
>> > >> 'readonly'@'10.86.29.32'
>> > >> (using password: NO)"
>> > >>
>> > >> I'd advise reading your stack traces a bit more carefully. You should
>> > >> check your permissions or if you don't own the DB, check with your DBA
>> > >> to find out what user you should use to access your DB.
>> > >>
>> > >> - Amit
>> > >>
>> > >> On Mon, Oct 29, 2012 at 9:38 PM, kunal sachdeva
>> > >>  wrote:
>> > >>> Hi,
>> > >>>
>> > >>> I have tried using data-import in my local system. I was able to
>> > execute
>> > >> it
>> > >>> properly. but when I tried to do it unix server I got following
>> error:-
>> > >>>
>> > >>>
>> > >>> INFO: Starting Full Import
>> > >>> Oct 30, 2012 9:40:49 AM
>> > >>> org.apache.solr.handler.dataimport.SimplePropertiesWriter
>> > >>> readIndexerProperties
>> > >>> WARNING: Unable to read: dataimport.properties
>> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.update.DirectUpdateHandler2
>> > >>> deleteAll
>> > >>> INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
>> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
>> onInit
>> > >>> INFO: SolrDeletionPolicy.onInit: commits:num=1
>> > >>>
>> > >>>
>> > >>
>> >
>> commit{dir=/opt/testsolr/multicore/core0/data/index,segFN=segments_1,version=1351490646879,generation=1,filenames=[segments_1]
>> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
>> > >>> updateCommits
>> > >>> INFO: newest commit = 1351490646879
>> > >>> Oct 30, 2012 9:40:49 AM
>> > >> org.apache.solr.handler.dataimport.JdbcDataSource$1
>> > >>> call
>> > >>> INFO: Creating a connection for entity destination with URL:
>> > >> jdbc:mysql://
>> > >>> 172.16.37.160:3306/hpcms_db_new
>> > >>> Oct 30, 2012 9:40:50 AM org.apache.solr.common.SolrException log
>> > >>> SEVERE: Exception while processing: destination document :
>> > >>>
>> > >>
>> >
>> SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
>> > >>> Unable to execute query: select name,id from hp_city Processing
>> > Document
>> > >> # 1
>> > >>>at
>> > >>>
>> > >>
>> >
>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)
>> > >>>at
>> > >>>
>> > >>
>> >
>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
>> > >>>at
>> > >>>
>> > >>
>> >
>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
>> > >>>at
>> > >>>
>> > >>
>> >
>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)
>> > >>> Caused by: java.lang.RuntimeException:
>> > >>> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable
>> > to
>> > >>> execute query: 

Re: Forwardslash delimiter.Solr4.0 query for path like /Customer/Content/*

[Jack Krupansky]

Maybe a custom search component that runs before the QueryComponent and does 
the escaping?


-- Jack Krupansky

-Original Message- 
From: Dirk Högemann

Sent: Tuesday, October 30, 2012 1:07 PM
To: solr-user@lucene.apache.org
Subject: Forwardslash delimiter.Solr4.0 query for path like 
/Customer/Content/*


Hi,

I am currently upgrading from Solr 3.5 to Solr 4.0

I used to have filter-bases restrictions for my search based on the paths
of documents in a content repository.
E.g.  fq={!q.op=OR df=}folderPath_}/customer/content/*

Unfortunately this does not work anymore, as lucene now supports
Regexpsearches - delimiting the expression with forward slashes:
http://lucene.apache.org/core/4_0_0-BETA/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Regexp_Searches

this leads to a parsed query, which is of course not what is intended:

RegexpQuery(folderPath_:/standardlsg/)
folderPath_:shareddocs RegexpQuery(folderPath_:/personen/)
folderPath_:*

Is there a possibility to make the example query above work, without
escaping the "/" with "\/"?
Otherwise I will have to parse all queries  (coming from persisted
configurations in the repositiory) and escape the relevant parts of the
queries on that field, which is somewhat ugly...

The field I search on is of type:


   
   
 
 
   
   


Best and thanks for any hints
Dirk 

Re: how to display MathML in search results?

[Joe Corneli]

On Tue, Oct 30, 2012 at 5:55 PM, Paul Libbrecht  wrote:
> Joe,
>
> if XHTML works fine... why would MathML not?

To clarify, it works fine when displaying nodes, not when displaying
search results (nor when browsing the select?q=... directly in Solr.

Joe

Re: row.get() in script transformer adds square brackets [] to string value

[Jack Krupansky]

The question is whether your field in your Solr schema has 
multiValued="true". If so, your JavaScript would see an array of values.


-- Jack Krupansky

-Original Message- 
From: Radek Zajkowski

Sent: Tuesday, October 30, 2012 7:44 AM
To: solr-user@lucene.apache.org
Subject: Re: row.get() in script transformer adds square brackets [] to 
string value


Thanks and sorry, forgot to include XML

In data config:



The Xml node:



Is the the CDATA section causing this?

Radek.

On Mon, Oct 29, 2012 at 9:10 PM, Jack Krupansky 
wrote:



Sounds like it is multivalued - the square brackets indicate an array.

-- Jack Krupansky

-Original Message- From: Radek Zajkowski
Sent: Monday, October 29, 2012 8:37 PM
To: solr-user@lucene.apache.org
Subject: row.get() in script transformer adds square brackets [] to string
value


Hi all,

would you know why I get (notice square brackets)

[1969 Harley Davidson Ultimate Chopper]

not

1969 Harley Davidson Ultimate Chopper

when calling

var description = row.get("ProductName").**toString();

in a script transformer?

Thank you,

Radek.

Re: Query regarding solr search

[Jack Krupansky]

-- Jack Krupansky

-Original Message- 
From: Leena Jawale

Sent: Tuesday, October 30, 2012 7:27 AM
To: solr-user@lucene.apache.org
Cc: Sawant, Amit2
Subject: RE: Query regarding solr search

Hi,

I understand that in the solr search considering the field type as text_en 
and not as int for the fields.
So how do I convert the field type of a particular field in solr XML as int 
so that I can operate that

field for range queries in solr??

Thanks,
Leena Jawale

From: Leena Jawale
Sent: Tuesday, October 30, 2012 12:18 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: Query regarding solr search

Hi,

I have created Solr XML data source. And on that I am working on less than 
operator.
I tried q=SerialNo:[ * TO 500 ].But It is showing records having 
SerialNo=1000. Could you help me this.?



Thanks,
Leena Jawale


The contents of this e-mail and any attachment(s) may contain confidential 
or privileged information for the intended recipient(s). Unintended 
recipients are prohibited from taking action on the basis of information in 
this e-mail and using or disseminating the information, and must notify the 
sender and delete it from their system. L&T Infotech will not accept 
responsibility or liability for the accuracy or completeness of, or the 
presence of any virus or disabling code in this e-mail" 

Re: Urgent Help Needed: Solr Data import problem

[Travis Low]

We're getting a little far afield...but here is the incantation:

mysql> grant all on DBNAME.* to 'USER'@'IP-ADDRESS' identified by
'PASSWORD';
mysql> flush privileges;

cheers,

Travis

On Tue, Oct 30, 2012 at 2:40 PM, Amit Nithian  wrote:

> This error is typically because of a mysql permissions problem. These
> are usually resolved by a GRANT statement on your DB to allow for
> users to connect remotely to your database server.
>
> I don't know the full syntax but a quick search on Google should yield
> what you are looking for. If you don't control access to this DB, talk
> to your sys admin who does maintain this access and s/he should be
> able to help resolve this.
>
> On Tue, Oct 30, 2012 at 7:13 AM, Travis Low  wrote:
> > Like Amit said, this appears not to be a Solr problem. From the command
> > line of your machine, try this:
> >
> > mysql -u'readonly' -p'readonly' -h'10.86.29.32' hpcms_db_new
> >
> > If that works, and 10.86.29.32 is the server referenced by the URL in
> your
> > data-config.xml problem, then at least you know you have database
> > connectivity, and to the right server.
> >
> > Also, if your unix server (presumably your mysql server) is 10.86.29.32,
> > then the URL in your data-config.xml is pointing to the wrong machine.
>  If
> > the one in the data-config.xml is correct, you need to test for
> > connectivity to that machine instead.
> >
> > cheers,
> >
> > Travis
> >
> > On Tue, Oct 30, 2012 at 5:15 AM, kunal sachdeva <
> kunalsachde...@gmail.com>wrote:
> >
> >> Hi,
> >>
> >> This is my data-config file:-
> >>
> >> 
> >>
> >>   
> >>
> >>   
> >>
> >>  >> name="package" query="select concat('pckg', id) as
> id,pkg_name,updated_time
> >> from hp_package_info;">
> >> 
> >>
> >>  >> name="destination"
> >>  query="select name,id from hp_city">
> >>  
> >> 
> >> 
> >>   
> >> 
> >>
> >>
> >> and password is not null. and 10.86.29.32 is my unix server ip.
> >>
> >> regards,
> >> kunal
> >>
> >> On Tue, Oct 30, 2012 at 2:42 PM, Dave Stuart 
> wrote:
> >>
> >> > It looks as though you have a password set on your unix server. you
> will
> >> > need to either remove this or ti add the password into the connection
> >> string
> >> >
> >> > e.g. readonly:[yourpassword]@'10.86.29.32'
> >> >
> >> >
> >> >
> >> > >> 'readonly'@'10.86.29.32'
> >> > >> (using password: NO)"
> >> > On 30 Oct 2012, at 09:08, kunal sachdeva wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > I'm not getting this error while running in local machine. Please
> Help
> >> > >
> >> > > Regards,
> >> > > Kunal
> >> > >
> >> > > On Tue, Oct 30, 2012 at 10:32 AM, Amit Nithian 
> >> > wrote:
> >> > >
> >> > >> This looks like a MySQL permissions problem and not a Solr problem.
> >> > >> "Caused by: java.sql.SQLException: Access denied for user
> >> > >> 'readonly'@'10.86.29.32'
> >> > >> (using password: NO)"
> >> > >>
> >> > >> I'd advise reading your stack traces a bit more carefully. You
> should
> >> > >> check your permissions or if you don't own the DB, check with your
> DBA
> >> > >> to find out what user you should use to access your DB.
> >> > >>
> >> > >> - Amit
> >> > >>
> >> > >> On Mon, Oct 29, 2012 at 9:38 PM, kunal sachdeva
> >> > >>  wrote:
> >> > >>> Hi,
> >> > >>>
> >> > >>> I have tried using data-import in my local system. I was able to
> >> > execute
> >> > >> it
> >> > >>> properly. but when I tried to do it unix server I got following
> >> error:-
> >> > >>>
> >> > >>>
> >> > >>> INFO: Starting Full Import
> >> > >>> Oct 30, 2012 9:40:49 AM
> >> > >>> org.apache.solr.handler.dataimport.SimplePropertiesWriter
> >> > >>> readIndexerProperties
> >> > >>> WARNING: Unable to read: dataimport.properties
> >> > >>> Oct 30, 2012 9:40:49 AM
> org.apache.solr.update.DirectUpdateHandler2
> >> > >>> deleteAll
> >> > >>> INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
> >> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
> >> onInit
> >> > >>> INFO: SolrDeletionPolicy.onInit: commits:num=1
> >> > >>>
> >> > >>>
> >> > >>
> >> >
> >>
> commit{dir=/opt/testsolr/multicore/core0/data/index,segFN=segments_1,version=1351490646879,generation=1,filenames=[segments_1]
> >> > >>> Oct 30, 2012 9:40:49 AM org.apache.solr.core.SolrDeletionPolicy
> >> > >>> updateCommits
> >> > >>> INFO: newest commit = 1351490646879
> >> > >>> Oct 30, 2012 9:40:49 AM
> >> > >> org.apache.solr.handler.dataimport.JdbcDataSource$1
> >> > >>> call
> >> > >>> INFO: Creating a connection for entity destination with URL:
> >> > >> jdbc:mysql://
> >> > >>> 172.16.37.160:3306/hpcms_db_new
> >> > >>> Oct 30, 2012 9:40:50 AM org.apache.solr.common.SolrException log
> >> > >>> SEVERE: Exception while processing: destination document :
> >> > >>>
> >> > >>
> >> >
> >>
> SolrInputDocument[{}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
> >> > >>> Unable to execute query: select name,id from hp_city Processing
> >> > Document
> >> > >> # 1
> >> > >>>at
> >> > >>>
> >> > >>
> >> >
> >>
> org.apache.solr.handler.d

Re: how to display MathML in search results?

[Paul Libbrecht]

Le 30 oct. 2012 à 20:30, Joe Corneli a écrit :
>  select?q=... directly in Solr.

What's in there?
Are MathML islands gone?

paul

Re: how to display MathML in search results?

[Joe Corneli]

On Tue, Oct 30, 2012 at 8:21 PM, Paul Libbrecht  wrote:
> Le 30 oct. 2012 à 20:30, Joe Corneli a écrit :
>>  select?q=... directly in Solr.
>
> What's in there?
> Are MathML islands gone?

Yep!  Like this:


[...]
For example, the universal closure of x + 2 = 3 x 2 3 x+2=3 is
[...]


instead of

For example, the universal closure of x+2=3x23x+2=3

Re: Tokenizer question

[Jack Krupansky]

Maybe it would be simplest to use a PatternReplaceCharFilter to eliminate 
the ".jpg", and then use the StandardTokenizer, or use the white space 
tokenizer and the Word Delimiter Filter.


-- Jack Krupansky

-Original Message- 
From: RL

Sent: Tuesday, October 30, 2012 3:57 AM
To: solr-user@lucene.apache.org
Subject: Tokenizer question

I could not find a solution to that in the documentation or the mailing 
list,

so here's my question.

I have files following the pattern: firstname_lastname_employeenumber.jpg

I'm able to search for the single terms firstname or lastname or the
employeenumber using a solr.PatternTokenizerFactory. Where I split at
underscore and dot.

But, now I also want to search for firstname_lastname or
lastname_employeenumber
Which does not work because the underscore was tokenized and is not part of
the indexed token anymore.


Any suggestions how to do that?

Thanks in advance.

RL



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Tokenizer-question-tp4016932.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Re: how to display MathML in search results?

[Paul Libbrecht]

So I guess 

> We call hyperbolic a loxodromic transformation that has a single 
> fixed point.


Also becomes

We call hyperbolic a loxodromic transformation that has a single fixed point.

?

In this case, it's definitely the Drupal side doing the html-stripping.

paul

Le 30 oct. 2012 à 21:51, Joe Corneli a écrit :

> On Tue, Oct 30, 2012 at 8:21 PM, Paul Libbrecht  wrote:
>> Le 30 oct. 2012 à 20:30, Joe Corneli a écrit :
>>> select?q=... directly in Solr.
>> 
>> What's in there?
>> Are MathML islands gone?
> 
> Yep!  Like this:
> 
> 
> [...]
> For example, the universal closure of x + 2 = 3 x 2 3 x+2=3 is
> [...]
> 
> 
> instead of
> 
> For example, the universal closure of  id="p12.1.m1.1" display="inline" xref="p12.1.m1.1.cmml"> id="p12.1.m1.1a" xref="p12.1.m1.1.cmml"> xref="p12.1.m1.1.6.cmml"> xref="p12.1.m1.1.6.1.cmml"> xref="p12.1.m1.1.1.cmml">x xref="p12.1.m1.1.2.cmml">+ xref="p12.1.m1.1.3.cmml">2 xref="p12.1.m1.1.4.cmml">= xref="p12.1.m1.1.5.cmml">3 id="p12.1.m1.1.cmml" encoding="MathML-Content"
> xref="p12.1.m1.1"> xref="p12.1.m1.1.6"> xref="p12.1.m1.1.4"/> xref="p12.1.m1.1.6.1"> xref="p12.1.m1.1.2"/> xref="p12.1.m1.1.1">x xref="p12.1.m1.1.3">2 id="p12.1.m1.1.5.cmml"
> xref="p12.1.m1.1.5">3 id="p12.1.m1.1b" encoding="application/x-tex"
> xref="p12.1.m1.1.cmml">x+2=3

Re: how to display MathML in search results?

[Joe Corneli]

On Tue, Oct 30, 2012 at 9:16 PM, Paul Libbrecht  wrote:
> So I guess
>
>> We call hyperbolic a loxodromic transformation that has a single 
>> fixed point.
>
> Also becomes
>
> We call hyperbolic a loxodromic transformation that has a single fixed point.

Indeed, that does seem to be the case!

Re: Are there any limitations on multi-value field joins?

[Erick Erickson]

Whenever anyone starts talking about using Solr to perform what
would be multi-way DB joins I break out in hives.

First of all, the "limited join capability" in Solr only returns the
values from ONE of the documents. There's no way to return values
from both the from and to documents.

Second, Solr's join capability is a poor fit if the fields being joined have
many unique values, so that's something to be careful of

I'd advise that you see if you can flatten (de-normalize) your data such
that you can make simple queries rather than try to use Solr like you
would a DB...

FWIW,
Erick

On Tue, Oct 30, 2012 at 7:20 AM, Steven Livingstone Pérez
 wrote:
> Hi - I've done quite a bit of Googling and reading but can't find a 
> definitive answer to this.
> I would like to have a list of key data rows each with a unique id and some 
> data.
> datarow1 a b cdatarow2 x y zdatarow3 m n o...
> I'd then like to have other rows that point to one or more of they data rows 
> that have a multi-valued field that can contain one or many of the unique 
> id's above.
> User1 datarow1, datarow2, datarow3 etcUser2 datarow4, datarow21, datarow43 
> etc...
> Then i will join from the user1 row to the data row.
> My question is simply are there *any* limitation on doing this kind of join? 
> I believe there are some geo-spatial issues and sorting (i don't need to sort 
> on the id) but before i jump fully into this approach i've like to understand 
> anything i may run into - or whether it is better to have them as individual 
> rows and join them that way.
> many thanks,/steven

RE: Are there any limitations on multi-value field joins?

[Steven Livingstone Pérez]

Thanks. Can you explain a bit more about your second point below. Specifically 
what makes it a bad fit? (design wise, performance)?

Thanks again.
Steven

Sent from my Windows Phone

From: Erick Erickson
Sent: 30/10/2012 22:22
To: solr-user@lucene.apache.org
Subject: Re: Are there any limitations on multi-value field joins?

Whenever anyone starts talking about using Solr to perform what
would be multi-way DB joins I break out in hives.

First of all, the "limited join capability" in Solr only returns the
values from ONE of the documents. There's no way to return values
from both the from and to documents.

Second, Solr's join capability is a poor fit if the fields being joined have
many unique values, so that's something to be careful of

I'd advise that you see if you can flatten (de-normalize) your data such
that you can make simple queries rather than try to use Solr like you
would a DB...

FWIW,
Erick

On Tue, Oct 30, 2012 at 7:20 AM, Steven Livingstone Pérez
 wrote:
> Hi - I've done quite a bit of Googling and reading but can't find a 
> definitive answer to this.
> I would like to have a list of key data rows each with a unique id and some 
> data.
> datarow1 a b cdatarow2 x y zdatarow3 m n o...
> I'd then like to have other rows that point to one or more of they data rows 
> that have a multi-valued field that can contain one or many of the unique 
> id's above.
> User1 datarow1, datarow2, datarow3 etcUser2 datarow4, datarow21, datarow43 
> etc...
> Then i will join from the user1 row to the data row.
> My question is simply are there *any* limitation on doing this kind of join? 
> I believe there are some geo-spatial issues and sorting (i don't need to sort 
> on the id) but before i jump fully into this approach i've like to understand 
> anything i may run into - or whether it is better to have them as individual 
> rows and join them that way.
> many thanks,/steven

Re: SolrJ 4.0.0 addFilterQuery() issue ?

[Chris Hostetter]

: org.apache.solr.common.SolrException:
: org.apache.lucene.queryparser.classic.ParseException: Cannot parse
: '[Ljava.lang.String;@1ec278b5': Encountered "" at line 1, column 28.

Hmmm.. that looks like a pretty anoying bug -- somehwere SolrJ is 
serializing a String[] instead of sending the individual String values.

can you please open a jira for this with these details?

: Is there a new/alternate way in SolrJ 4 that this is done ?

I would say that one possible workarround may be to 
use...
solrQuery.add("fq", "your first filter");
solrQuery.add("fq", "your second filter");

...but i don't know where the bug is to know if that will actally work.  
if you could try that also and mention the results in a comment in the 
Jira you open that would be helpful.

-Hoss

Re: Subqueries... using results from the main query

[Otis Gospodnetic]

Hi,

Wouldn't this work for you:

q=1234 +product_id:X +product_id:Y

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html


On Tue, Oct 30, 2012 at 9:23 AM, Gustav  wrote:
> Hello everyone,
>
> i would like to know if it is possible to make a SUBQUERY inside my query
> considering a field's current result of the main query in this subquery,
> Let me try to give a better explaination
>
> I Have a simple query, something like this: /?q="1234"&qf=product_id, this
> returns to me several fields, including:
>
> 14114 (this is the id of the related
> product)
> 14115
>
> I want to know if, in this same query, is possible to return the products
> wich id are 14114 and 14115, instead of the normal results...
>
> Thanks in advance,
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Subqueries-using-results-from-the-main-query-tp4017000.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Http 500/503 Responses from Solr

[deniz]

well, as for details for the server, it is running on a server with 6 gigs of
ram ( jvm heap max is 4 gb), using ram directory for the index. 

here is my config file (some values are changed as i have been testing):


  LUCENE_40
  
  
  
  
  
  
  
  
  
  ${solr.data.dir:}
  
  
  
  
  
 
   15000
   false
 

  ${solr.data.dir:}

  
  
1024



true
   20
   200

  
  


  

 static firstSearcher warming in
solrconfig.xml

  

true
2
  
  


  
  
 
   explicit
   10
   text
 

  
 
   explicit
   json
   true
   text
 
  
  
 
   true
   json
   true
 
  
  
 
   explicit
   velocity
   browse
   layout
   Solritas
   edismax
   
  text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
  title^10.0 description^5.0 keywords^5.0 author^2.0
resourcename^1.0
   
   text
   100%
   *:*
   10
   *,score
   
 text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
 title^10.0 description^5.0 keywords^5.0 author^2.0 resourcename^1.0
   
   text,features,name,sku,id,manu,cat,title,description,keywords,author,resourcename
   3
   on
   cat
   manu_exact
   content_type
   author_s
   ipod
   GB
   1
   cat,inStock
   after
   price
   0
   600
   50
   popularity
   0
   10
   3
   manufacturedate_dt
   NOW/YEAR-10YEARS
   NOW
   +1YEAR
   before
   after
   on
   content features title name
   html
   
   
   0
   title
   0
   name
   3
   200
   content
   750
   on
   false
   5
   2
   5
   true
   true
   5
3
 
 
   spellcheck
 
  
  
  
  

  true
  ignored_
  true
  links
  ignored_

  
  
  
  
  

  solrpingquery


  all

  
  

 explicit
 true

  

  
textSpell

  default
  name
  solr.DirectSolrSpellChecker
  internal
  0.5
  2
  1
  5
  4
  0.01


  wordbreak
  solr.WordBreakSolrSpellChecker
  name
  true
  true
  10

  
  

  text
  default
  wordbreak
  on
  true
  10
  5
  5
  true
  true
  10
  5


  spellcheck

  
  
  

  text
  true


  tvComponent

  
  

  default
  org.carrot2.clustering.lingo.LingoClusteringAlgorithm
  20
  clustering/carrot2
   ENGLISH


  stc
  org.carrot2.clustering.stc.STCClusteringAlgorithm

  
  

  true
  default
  true
  name
  id
   features
   true
   false
   edismax
   
 text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
   
   *:*
   10
   *,score


  clustering

  
  
  
 
  true


  terms

  
  

  

  100

  
  

  70
  0.5
  [-\w ,/\n\"']{20,200}

  
  

  
  

  
  
  
  
  
  
  
  

  
  

  

  

  10
  .,!? 	


  

  

  WORD
  en
  US

  

  
  
text/plain; charset=UTF-8
  

  
5
  
  
*:*
  





and this is how i start solr :

/path/to/java \
-server \
-XX:+UnlockExperimentalVMOptions \
-XX:+UseG1GC \
-XX:+UseCompressedStrings \
-Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=8889 \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false \
-Djava.rmi.server.hostname=myhost \
-Xms1024m \
-Xmx4096m \
-d64 \
-jar start.jar &


and this is my jetty.xml

http://www.eclipse.org/jetty/configure.dtd";>


  org.eclipse.jetty.server.Request.maxFormContentSize
  20


  
10
1
false
  


  
  


5
1500
false
  
  


  

 
   
 
   
   
 
   
   
 
   
 

  

true
false
false
1000
false
false

  

   

  
  
   
org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern
.*/servlet-api-[^/]*\.jar$
  

  


  

  
/contexts
0
  

Re: Stemming of irregular plurals such as Women and Skies

[Otis Gospodnetic]

Hi Raj,

I think you are looking for StemmerOverrideFilterFactory :

http://wiki.apache.org/solr/LanguageAnalysis#solr.StemmerOverrideFilterFactory

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html


On Tue, Oct 30, 2012 at 12:54 PM, Nemani, Raj  wrote:
> All,
> Any tips that the user group can provide on a best method do deal with these 
> kind of the words.   I do understand that this is a special situation and 
> found a good (I thought) general explanation in the following post from 
> Stackoverflow.  But from an implementation standpoint I was wondering if 
> anybody has dealt with  these sort of situations and found an optimal method. 
>  We are using Porter stemmer.
>
> http://stackoverflow.com/questions/10298231/difference-between-word-stemming-and-depluralization
>
> I do appreciate any help.
>
> Thanks
> Raj
>

Re: [Announce] Apache Solr 4.0 with RankingAlgorithm 1.4.4 and Realtime NRT available for download

[Otis Gospodnetic]

Hi,

Unless I'm missing something, Nagendra is now (finally - good to see
the contrib Nagendra!) trying to provide the patch for Solr.  If the
patch goes in, then there is no need for Nagendra to have this
external project any more.

Nagendra, am I interpreting your intentions correctly here?

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html


On Mon, Oct 29, 2012 at 12:17 PM, Michael Della Bitta
 wrote:
> As an external observer, I think the main problem is your branding.
> "Realtime Near Realtime" is definitely an oxymoron, and your ranking
> algorithm is called "Ranking Algorithm," which is generic enough to
> suggest that a. it's the only ranking algorithm available, and b. by
> implication, that Solr doesn't have one built in.
>
> I would suggest two improvements:
>
> 1. Come up with a top-level name for your overall efforts. Apache
> Foundation has 'Apache,' so automatic branding of every component they
> build. Then your ranking algorithm could be called "Tgels Ranking
> Algorithm for Apache Solr" (for example), which is totally legit. And
> "Tgels Realtime Search for Apache Solr."
>
> 2. Maybe point out that you're building on top of the work of the
> Apache Solr and Lucene projects a little more prominently.
>
> I think with those two little tweaks, you'd actually very easily get
> more people interested in your contributions.
>
> Just my two cents,
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Mon, Oct 29, 2012 at 11:35 AM, Nagendra Nagarajayya
>  wrote:
>>
>> Jack:
>>
>> I respect your hard-work responding to user problems on the mail list. So it
>> would be nicer to try out Realtime NRT then pass rogue comments, whether a
>> contribution is legit/spam or a scam... I guess it illuminates the narrow
>> minded view of oneself ...  The spirit of open source is contributions from
>> not only commiters but other developers, from the Solr wiki "A half-baked
>> patch in Jira, with no documentation, no tests and no backwards
>> compatibility is better than no patch at all."
>>
>> You would gain more respect if you actually download realtime-nrt, check out
>> if it does provide a view of a realtime index compared to a  point-in-time
>> snapshot, see if you can understand the code and provide clarity  and
>> feedback to the list if you do find problems with it. realtime-nrt offers
>> search capability as to realtime-get. Checkout if  this is true ... I would
>> really welcome your comments on the list or through the JIRA here:
>>
>> https://issues.apache.org/jira/browse/SOLR-3816
>>
>>
>> Regards,
>>
>> Nagendra Nagarajayya
>> http://solr-ra.tgels.org
>> http://rankingalgorithm.tgels.org
>>
>> On 10/29/2012 7:30 AM, Jack Krupansky wrote:
>>>
>>> Could any of the committers here confirm whether this is a legitimate
>>> effort? I mean, how could anything labeled "Apache ABC with XYZ" be an
>>> "external project" and be sanctioned/licensed by Apache? In fact, the linked
>>> web page doesn't even acknowledge the ownership of the Apache trademarks or
>>> ASL. And the term "Realtime NRT" is nonsensical. Even worse: "Realtime NRT
>>> makes available a near realtime view". Equally nonsensical. Who knows, maybe
>>> it is legit, but it sure comes across as a scam/spam.
>>>
>>> -- Jack Krupansky
>>>
>>> -Original Message- From: Nagendra Nagarajayya
>>> Sent: Monday, October 29, 2012 10:06 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: [Announce] Apache Solr 4.0 with RankingAlgorithm 1.4.4 and
>>> Realtime NRT available for download
>>>
>>> Hi!
>>>
>>> I am very excited to announce the availability of Apache Solr 4.0 with
>>> RankingAlgorithm 1.4.4 and Realtime NRT. Realtime NRT is a high
>>> performance and more granular NRT implementation as to soft commit. The
>>> update performance is about 70,000 documents / sec* (almost 1.5-2x
>>> performance improvement over soft-commit). You can also scale up to 2
>>> billion documents* in a single core, and query half a billion documents
>>> index in ms**. Realtime NRT is different from realtime-get. realtime-get
>>> does not have search capability and is a lookup by id. Realtime NRT
>>> allows full search, see here 
>>> for more info.
>>>
>>> Realtime NRT has been contributed back to Solr, see JIRA:
>>> https://issues.apache.org/jira/browse/SOLR-3816
>>>
>>> RankingAlgorithm 1.4.4 supports the entire Lucene Query Syntax, ą and/or
>>> boolean/dismax/boost queries and is compatible with the new Lucene 4.0
>>> api.
>>>
>>> You can get more information about Solr 4.0 with RankingAlgorithm 1.4.4
>>> and Realtime NRT performance from here:
>>> http://solr-ra.tgels.org/wiki/en/Near_Real_Time_Search_ver_4.x
>>>
>>> You can download Solr 4.0 with RankingAlgorithm 1.4.4 from here:

Re: SolrCloud AutoSharding? In enterprise environment?

[Otis Gospodnetic]

Hi Joseph,

I think that if your clients talk to the VIP and not directly to Solr
via 
http://search-lucene.com/jd/solr/solr-solrj/org/apache/solr/client/solrj/impl/CloudSolrServer.html
you lose the ability to hit leaders directly (CloudSolrServer knows
about them), which translates to a bit more work for Solr.

You should be able to specify just the # of replicas and shards and
not worry about how sharding will be done.
See
https://issues.apache.org/jira/browse/SOLR-3894
https://issues.apache.org/jira/browse/SOLR-2592
http://search-lucene.com/?q=index+query+routing+shards&fc_project=Solr

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html


On Tue, Oct 30, 2012 at 12:07 PM, joseph_12345
 wrote:
> Hi All,
>
> I have looked at the below post URL and it really helps me a lot.
>
> http://lucene.472066.n3.nabble.com/SolrCloud-AutoSharding-td4011834.html
>
> But I have few questions.
>
> We are exploring SolrCloud to index millions of product details. We are
> planning to use a complete pool for SollCloud with a set of 15 physical
> machines. The clients will be hitting the VIP URL of SolrCloud pool instead
> of individual machine names or IP addresses. So my questions are,
>
> 1. While indexing the product details, do I need to take care of custom
> sharding strategy or I just need to specify the number of Shards as 15(total
> number of boxes in pool) and the SolrCloud takes care of sharding
> internally?
>
> 2. If SolrCloud takes care of sharding since this shards are in 15 different
> boxes, does SolrCloud internally do full scan to all these boxes if client
> is querying by specifying the VIP url (Not the individual machine names)? Or
> does SolrCloud do some extra intelligent logic to avoid full scan?
>
> 3. I couldn't find proper documentation about what SolrCloud does internally
> in terms of Sharding and what developer has to do to optimize queries?
>
> Any response on this is greatly appreciated.
>
> Thanks
> Joseph
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/SolrCloud-AutoSharding-In-enterprise-environment-tp4017036.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr question

[Otis Gospodnetic]

Hi/bok Mladen,

Could function queries help you here?
Have a look at http://wiki.apache.org/solr/FunctionQuery

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html


On Tue, Oct 30, 2012 at 10:26 AM, mkaran  wrote:
> I have a question about integrating custom functionality into Solr.
>
> What i want is the following scenario:
>  1. A query is issued to Solr
>  2. Solr uses my custom method to calculate the score (relevance) of each
>  document with respect to the query.
>  3. the results are returned sorted by descending score.
>
> I am unsure how to implement this.
> From the lucene documentation i thought it would be enough to implement
> an extension of the Similarity class. Now im not so sure (i don't want to
> just
> override tf or idf i need a completely different scoring method, also i want
> the
> method to be applied to all documents and not only those that pass an
> initial
> boolean filter).
>
> Which classes would i have to implement to get this behaviour?
> Is this a good start?
> http://lucene.apache.org/core/old_versioned_docs/versions/3_5_0/api/core/org/apache/lucene/search/package-summary.html#scoring)
> Is there maybe a simpler way of achieving this?
>
> Thanks for any advice,
> Mladen
>
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-question-tp4017024.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: throttle segment merging

[Otis Gospodnetic]

Hi Radim,

To address your comment about JIRA and search - perhaps this is better
- and it finds you:
http://search-lucene.com/?q=throttle+merge&fc_project=Solr

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html



On Mon, Oct 29, 2012 at 6:57 PM, Radim Kolar  wrote:
> Dne 29.10.2012 12:18, Michael McCandless napsal(a):
>
>> With Lucene 4.0, FSDirectory now supports merge bytes/sec throttling
>> (FSDirectory.setMaxMergeWriteMBPerSec): it rate limits that max
>> bytes/sec load on the IO system due to merging.
>>
>> Not sure if it's been exposed in Solr / ElasticSearch yet ...
>
> its not available in solr. Also solr class hierarchy for directory providers
> is bit different from lucene. In solr, MMAP DF and NIOFSDF needs to be
> subclass of StandardDF. then add write limit property to standardDF and it
> will be inherited by others like in lucene.
>
> solr
> http://lucene.apache.org/solr/4_0_0/solr-core/org/apache/solr/core/CachingDirectoryFactory.html
> lucene
> http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/store/FSDirectory.html

Query regarding solr search

[Leena Jawale]

Hi,
My XML file is



1001 
IBM 1001 
INFPW03772 
22/4/2010 
1000


1002 
DELL 1002 
INFPW03734 
09/7/2010 
5000



But still the SerialNo and price1 working according to type text.

Thanks,
Leena Jawale


From: Leena Jawale
Sent: Tuesday, October 30, 2012 4:58 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: RE: Query regarding solr search

Hi,

I understand that in the solr search considering the field type as text_en and 
not as int for the fields.
So how do I convert the field type of a particular field in solr XML as int so 
that I can operate that
field for range queries in solr??

Thanks,
Leena Jawale

From: Leena Jawale
Sent: Tuesday, October 30, 2012 12:18 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: Query regarding solr search

Hi,

I have created Solr XML data source. And on that I am working on less than 
operator.
I tried q=SerialNo:[ * TO 500 ].But It is showing records having SerialNo=1000. 
Could you help me this.?


Thanks,
Leena Jawale


The contents of this e-mail and any attachment(s) may contain confidential or 
privileged information for the intended recipient(s). Unintended recipients are 
prohibited from taking action on the basis of information in this e-mail and 
using or disseminating the information, and must notify the sender and delete 
it from their system. L&T Infotech will not accept responsibility or liability 
for the accuracy or completeness of, or the presence of any virus or disabling 
code in this e-mail"

Re: Query regarding solr search

[Jack Krupansky]

Okay,  is correct, but it belongs in 
your Solr schema file, schema.xml, NOT your Solr XML data file. In other 
words, the schema.xml file declares the type for each field, NOT the Solr 
XML field elements.


-- Jack Krupansky

-Original Message- 
From: Leena Jawale

Sent: Wednesday, October 31, 2012 12:06 AM
To: 'solr-user@lucene.apache.org'
Cc: Sawant, Amit2
Subject: Query regarding solr search


Hi,
My XML file is



1001 
IBM 1001 
INFPW03772 
22/4/2010 
1000


1002 
DELL 1002 
INFPW03734 
09/7/2010 
5000



But still the SerialNo and price1 working according to type text.

Thanks,
Leena Jawale


From: Leena Jawale
Sent: Tuesday, October 30, 2012 4:58 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: RE: Query regarding solr search

Hi,

I understand that in the solr search considering the field type as text_en 
and not as int for the fields.
So how do I convert the field type of a particular field in solr XML as int 
so that I can operate that

field for range queries in solr??

Thanks,
Leena Jawale

From: Leena Jawale
Sent: Tuesday, October 30, 2012 12:18 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: Query regarding solr search

Hi,

I have created Solr XML data source. And on that I am working on less than 
operator.
I tried q=SerialNo:[ * TO 500 ].But It is showing records having 
SerialNo=1000. Could you help me this.?



Thanks,
Leena Jawale


The contents of this e-mail and any attachment(s) may contain confidential 
or privileged information for the intended recipient(s). Unintended 
recipients are prohibited from taking action on the basis of information in 
this e-mail and using or disseminating the information, and must notify the 
sender and delete it from their system. L&T Infotech will not accept 
responsibility or liability for the accuracy or completeness of, or the 
presence of any virus or disabling code in this e-mail" 

Re: Query regarding solr search

[Romita Saha]

Did you change it in schema.xml ??

Thanks and regards,
Romita 



From:   Leena Jawale 
To: "'solr-user@lucene.apache.org'" <'solr-user@lucene.apache.org'>, 
Cc: "Sawant, Amit2 " 
Date:   10/31/2012 12:07 PM
Subject:Query regarding solr search




Hi,
My XML file is



1001 
IBM 1001 
INFPW03772 
22/4/2010 
1000


1002 
DELL 1002 
INFPW03734 
09/7/2010 
5000



But still the SerialNo and price1 working according to type text.

Thanks,
Leena Jawale


From: Leena Jawale
Sent: Tuesday, October 30, 2012 4:58 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: RE: Query regarding solr search

Hi,

I understand that in the solr search considering the field type as text_en 
and not as int for the fields.
So how do I convert the field type of a particular field in solr XML as 
int so that I can operate that
field for range queries in solr??

Thanks,
Leena Jawale

From: Leena Jawale
Sent: Tuesday, October 30, 2012 12:18 PM
To: 'solr-user@lucene.apache.org'
Cc: 'Sawant, Amit2 '
Subject: Query regarding solr search

Hi,

I have created Solr XML data source. And on that I am working on less than 
operator.
I tried q=SerialNo:[ * TO 500 ].But It is showing records having 
SerialNo=1000. Could you help me this.?


Thanks,
Leena Jawale


The contents of this e-mail and any attachment(s) may contain confidential 
or privileged information for the intended recipient(s). Unintended 
recipients are prohibited from taking action on the basis of information 
in this e-mail and using or disseminating the information, and must notify 
the sender and delete it from their system. L&T Infotech will not accept 
responsibility or liability for the accuracy or completeness of, or the 
presence of any virus or disabling code in this e-mail"

Re: SolrJ 4.0.0 addFilterQuery() issue ?

[Indika Tantrigoda]

Thanks for the reply Chris.

Yes you are correct, SolrJ is serializing a String[] instead of the
separate String values.

Using solrQuery.add("fq", "your first filter"); and solrQuery.add("fq",
"your second filter"); has the same effect. Because it calls the add()
method in the ModifiableSolrParams.java class. (Similar to
solrQuery.setFilterQueries()).

Yes, I will open a Jira issue for this with more information.

Thanks,
Indika

On 31 October 2012 05:08, Chris Hostetter  wrote:

>
> : org.apache.solr.common.SolrException:
> : org.apache.lucene.queryparser.classic.ParseException: Cannot parse
> : '[Ljava.lang.String;@1ec278b5': Encountered "" at line 1, column
> 28.
>
> Hmmm.. that looks like a pretty anoying bug -- somehwere SolrJ is
> serializing a String[] instead of sending the individual String values.
>
> can you please open a jira for this with these details?
>
> : Is there a new/alternate way in SolrJ 4 that this is done ?
>
> I would say that one possible workarround may be to
> use...
> solrQuery.add("fq", "your first filter");
> solrQuery.add("fq", "your second filter");
>
> ...but i don't know where the bug is to know if that will actally work.
> if you could try that also and mention the results in a comment in the
> Jira you open that would be helpful.
>
> -Hoss
>

need help on solr search

[jchen2000]

Hi Solr experts,

Our documents as well as queries consist of 10 properties in a particular
order. Because of stringent requirements on search latency, we grouped them
into only 2 fields with 5 properties each (we may use just 1 field, field
number over 3 seems too slow), and each property value is split into
fixed-length terms (like n-gram, hopefully to save search time) and prefixed
with property name. What we want is to find out how similar the query is to
the documents by comparing terms. We can't use the default OR operator since
it's slow, we wanted to take advantage of the prefix and the defined order. 

My questions are:
1) Can we do this simply through solr configuration, and how if possible?
2) If we need to customize solr request handler or anything else, where to
start?

Thanks a lot!

Jeremy



--
View this message in context: 
http://lucene.472066.n3.nabble.com/need-help-on-solr-search-tp4017191.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to change the boost of fields in edismx at runtime

[Indika Tantrigoda]

Hi Saroj,

You could use the boost function in a FunctionQuery. Something similar to,

_val_:Title^10 and _val_:Keyword^2
_val_:Title^2 and _val_:Keyword^10

See
http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documentson
how to use the boost function.

Thanks,
Indika

On 31 October 2012 10:48, roz dev  wrote:

> Hi All
>
> I am wondering if there is any way in which we can change the relative
> boost of different fields in edismax, at the run-time.
>
> For example, I can define 2 fields in my edismax query
>
> -Title - boosted to 10
> -Keyword - Boosted to 2
>
>
> Then, If I find that results are not of my liking then I would like to
> change the boost as following
>
> - Title - boosted to 2
> -Keyword - boosted to 10
>
> Is there any way to change this boost, at run-time, without having to
> restart solr with new boosts in edismax?
>
> Any thoughts are much appreciated.
>
> Thanks
> Saroj
>

Re: hot shard concept

[Shawn Heisey]

On 10/30/2012 5:05 AM, Dmitry Kan wrote:

Hi Shawn,

Thanks for sharing your story. Let me get it right:

How do you keep the incremental shard slim enough over time, do you
periodically redistribute the documents from it onto cold shards? If yes,
how technically you do it: the Lucene low-level way or Solr / SolrJ way?


Warning: This email fits nicely into the tl;dr category.  I'm including 
entirely too much information because I'm not sure which bits you're 
really interested in.


My database and Solr index have two fields that contain unique values.  
Solr's unique key is what we call the tag_id (alphanumeric), but each 
document also has a MySQL autoincrement field called did, for document 
id, or possibly delete id, which is a tlong in the Solr schema.  The 
MySQL primary key is did.  I divvy up documents among the six cold 
shards by a mod on the crc32 hash (MySQL function) of the did field, my 
cold shards are numbered 0 through 5.  That crc32 hash is not indexed or 
stored in Solr, but now that I think about it, perhaps I should add it 
to the Solr-specific database view.


The did field is also where I look for my "split point" which marks the 
line between hot and cold.  Values less than or equal to the split point 
are in cold shards, values greater than the split point go in the hot shard.


Once an hour, my SolrJ build system gets MAX(did) from the database and 
stores it in a JRobin RRD.  Every night, I consult those values and do 
document counts against the database to pick a new split point.  Then I 
index documents between the old split point and the new split point into 
the cold shards, and if that succeeds, I delete the same DID range from 
the hot shard.  I wrote all the code that does this using the SolrJ API, 
storing persistent values in a MySQL database table.  I'm not aware of 
any shortcuts I could use.


Additional note: Full reindexes are accomplished with the dataimport 
handler, using the following SQL query.  For the hot shard, I pass in a 
modVal of 0,1,2,3,4,5 so that it gets all of the documents in the did range:


SELECT * FROM ${dataimporter.request.dataView}
WHERE (
  (
did > ${dataimporter.request.minDid}
AND did <= ${dataimporter.request.maxDid}
  )
  ${dataimporter.request.extraWhere}
) AND (crc32(did) % ${dataimporter.request.numShards})
  IN (${dataimporter.request.modVal})

Back when we first started with Solr 1.4.0, the build system was written 
in Perl (LWP::Simple) and did everything but deletes with the dataimport 
handler.  Deletes were done by query using xml and the /update handler.


Thanks,
Shawn