Re: Problem when i'm trying to search something

[Anon15]

Thanks for replying !

This is my Schema.xml. 








--
View this message in context: 
http://lucene.472066.n3.nabble.com/Problem-when-i-m-trying-to-search-something-tp4135045p4135427.html
Sent from the Solr - User mailing list archive at Nabble.com.

Fwd: Inconsistent response from Cloud Query

[Vineet Mishra]

Copying.
Community: Looking forward for your response.

-- Forwarded message --
From: Vineet Mishra 
Date: Mon, May 12, 2014 at 5:57 PM
Subject: Re: Inconsistent response from Cloud Query
To: solr-user@lucene.apache.org


Hi Shawn,

There is no recovery case for me, neither the commit is pending.
The case I am talking about is when I restart the Cloud all over again with
index already flushed to disk.

Thanks!


On Sun, May 11, 2014 at 10:17 PM, Shawn Heisey  wrote:

> On 5/9/2014 11:42 AM, Cool Techi wrote:
> > We have noticed Solr returns in-consistent results during replica
> recovery and not all replicas are in the same state, so when your query
> goes to a replica which might be recovering or still copying the index then
> the counts may differ.
> > regards,Ayush
>
> SolrCloud should never send requests to a replica that is recovering.
> If that is happening (which I think is unlikely), then it's a bug.
>
> If *you* send a request to a replica that is still recovering, I would
> expect SolrCloud to redirect the request elsewhere unless distrib=false
> is used.  I'm not sure whether that actually happens, though.
>
> Thanks,
> Shawn
>
>

Re: URLDataSource : indexing from other Solr servers

[Gora Mohanty]

On 12 May 2014 22:52, helder.sepulveda  wrote:
> Here is the data config:
>
> 
> 
>
> 
>  url="http://slszip11.as.homes.com/solr/select?q=*:*";
> processor="XPathEntityProcessor"
> forEach="/response/result/doc"
> transformer="DateFormatTransformer">
>  xpath="/response/result/doc/str[@name='batch_address']"/>
>  xpath="/response/result/doc/str[@name='batch_state']"/>
>  xpath="/response/result/doc/str[@name='batch_city']"/>
>  xpath="/response/result/doc/str[@name='batch_zip']"/>
>  xpath="/response/result/doc/long[@name='sz_id']"/>
> 
> 
> 

Hmm, see no issues here. Can you also share your Solr schema?
Is the URL accessible, and the results from Solr show properly when
loaded in a browser window? I cannot seem to reach slszip11.as.homes.com
but that could be because it is restricted to certain IPs.

Regards,
Gora

Re: Easises way to insatll solr cloud with tomcat

[Jack Krupansky]

Check out HDS from Heliosearch - it comes packaged with Tomcat, ready to go:

http://heliosearch.com/download.html

-- Jack Krupansky

-Original Message- 
From: Aman Tandon

Sent: Monday, May 12, 2014 8:23 AM
To: solr-user@lucene.apache.org
Subject: Re: Easises way to insatll solr cloud with tomcat

Can anybody help me out??

With Regards
Aman Tandon


On Mon, May 12, 2014 at 1:24 PM, Aman Tandon wrote:


Hi,

I tried to set up solr cloud with jetty which works fine. But in our
production environment we uses tomcat so i need to set up the solr cloud
with the tomcat. So please help me out to how to setup solr cloud with
tomcat on single machine.

Thanks in advance.

With Regards
Aman Tandon

Re: Join in solr to get data from two cores

[Kamal Kishore]

NO reply from anybody..seems strange ?


On Fri, May 9, 2014 at 9:47 AM, Kamal Kishore
wrote:

> Any updates guys ?
>
>
> On Thu, May 8, 2014 at 2:05 PM, Kamal Kishore  > wrote:
>
>> Dear Team,
>>
>> I have two solr cores. One containing products information and second has
>> customers points. I am looking at solr join to query on first product core
>> & boost the results based on customer points in second core. I am not able
>> to frame solr query for this.
>>
>> Moreover, solr is not allowing to get data from both the core.
>>
>>
>> With RegardsK
>>
>> Kamal Kishore
>>
>>
>>
>>
>

-- 
-


Follow IndiaMART.com  for latest updates on this 
and more:  
  Mobile 
Channel: 

Solrcore.properties variable question.

[Guido Medina]

Hi,

We have a couple of Solr servers acting as master and slave, and each 
server have the same amount of cores, we are trying to configure the 
solrcore.properties so that an script is able to add cores without 
changing the solrcore.properties using a hack like this:


enable.master=false
enable.slave=true
master_url=http://master_solr:8983/solr/${solr.core.name}

Our idea is to have solr.core.name to be the dynamic variable, but once 
we go to admin, the master URL is not showing the last part, is there a 
format error or something trivial I'm missing?


Thanks,

Guido.

Re: SolrCloud - Highly Reliable / Scalable Resources?

[Otis Gospodnetic]

Hi,

Re:
> we have suffered several issues which always seem quite problematic to
resolve.

Try grabbing the latest version if you can.  We identified a number of
issues in older SolrCloud versions when working on large client setups with
thousands of cores, but a lot of those issues have been fixes in the more
recent versions.

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/



On Mon, May 12, 2014 at 9:53 AM, Darren Lee  wrote:

> Hi everyone,
>
> We have been using Solr Cloud (4.4) for ~ 6 months now. Functionally its
> excellent but we have suffered several issues which always seem quite
> problematic to resolve.
>
> I was wondering if anyone in the community can recommend good resources /
> reading for setting up a highly scalable / highly reliable cluster. A lot
> of what I see in the solr documentation is aimed at small setups or is
> quite sparse.
>
> Dealing with topics like:
>
> * Capacity planning
>
> * Losing nodes
>
> * Voting panic
>
> * Recovery failure
>
> * Replication factors
>
> * Elasticity / Auto scaling / Scaling recipes
>
> * Exhibitor
>
> * Container configuration, concurrency limits, packet drop tuning
>
> * Increasing capacity without downtime
>
> * Scalable approaches to full indexing hundreds of millions of
> documents
>
> * External health check vs CloudSolrServer
>
> * Separate vs local zookeeper
>
> * Benchmarks
>
>
> Sorry, I know that's a lot to ask heh. We are going to run a project for a
> month or so soon where we re-write all our run books and do deeper testing
> on various failure scenarios and the above but any starting point would be
> much appreciated.
>
> Thanks all,
> Darren
>

Please add me to Contributors Group

[Gireesh C. Sahukar]

Hi,

I'd like to be added to the contributors group. My wiki username is gireesh

 

Thanks

Gireesh

 

Re: SWF content not indexed

[Mauro Gregorio Binetti]

Hi Ahmet,
thank you for your response... yes I think I need tika to do these job,
using it like an OCR.
I was trying to go deep inside it as yet.
Any other suggestion from the group is welcome.

Regads,
Mauro


On Sun, May 11, 2014 at 2:34 PM, Ahmet Arslan  wrote:

>
>
>
> Hi,
>
> Solr/lucene only deals with text. There are some other projects that
> extract text from rich documents.
> Solr-cell uses http://tika.apache.org for extraction. May be tika (or any
> other tool) already extracts text from swf?
>
>
> On Sunday, May 11, 2014 9:40 AM, Mauro Gregorio Binetti <
> maurogregorio.bine...@gmail.com> wrote:
> Hi guys,
> how can I make it possibile to index content of SWF files? I'm using Solr
> 3.6.0.
>
> Regards,
> Mauro
>

KeywordTokenizerFactory splits the string for the exclamation mark

[nativecoder]

Hi All

I have a following field settings in solr schema

Exact_Word*" omitPositions="true"
termVectors="false" omitTermFreqAndPositions="true" compressed="true"
type="string_ci" multiValued="false" indexed="true" stored="true"
required="false" omitNorms="true"/>







As you can see Exact_Email has the KeywordTokenizerFactory and that should
treat the string as it is.

But when I enter email with the following string
"d!sdasdsdwasd...@dsadsadas.edu" it splits the string to two. I was under
the impression that KeywordTokenizerFactory will treat the string as it is.
*!*
Following is the query debug result. There you can see it has split the word 
 "parsedquery":"+((DisjunctionMaxQuery((Exact_Email:d))
-DisjunctionMaxQuery((Exact_Email:sdasdsdwasd...@dsadsadas.edu)))~1)",

can someone please tell why it produce the query result as this 

If I put a string without the "!" sign as below, the produced query will be
as below

"parsedquery":"+DisjunctionMaxQuery((Exact_Email:testresu...@testdomain.com))",

I thought if the KeywordTokenizerFactory is applied then it should return
the exact string as it is

Please help me to understand what is going wrong here




--
View this message in context: 
http://lucene.472066.n3.nabble.com/KeywordTokenizerFactory-splits-the-string-for-the-exclamation-mark-tp4135460.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: ContributorsGroup add request

[Jim Martin]

Shawn-
   Thanks much. Icon ideas have been recorded.
-Jim

On 5/11/2014 10:39 AM, Shawn Heisey wrote:

On 5/10/2014 6:35 PM, Jim Martin wrote:

Please add me the ContributorsGroup; I've got some Solr icons I'd
like to suggest to the community. Perhaps down the road I can contribute
more. I'm the team lead at Overstock.Com for search, and Solr is the
foundation of what we do.

Username: JamesMartin

I went to add you, but someone else has already done so.

It's entirely possible that because of the Apache email outage, they
have already replied, but the message hasn't made it through to the list
yet.  I'm adding you as a CC here (which I normally don't do) so that
you'll get notified faster.

Thanks,
Shawn

Re: Join in solr to get data from two cores

[Kamal Kishore]

Any updates guys ?


On Thu, May 8, 2014 at 2:05 PM, Kamal Kishore
wrote:

> Dear Team,
>
> I have two solr cores. One containing products information and second has
> customers points. I am looking at solr join to query on first product core
> & boost the results based on customer points in second core. I am not able
> to frame solr query for this.
>
> Moreover, solr is not allowing to get data from both the core.
>
>
> With RegardsK
>
> Kamal Kishore
>
>
>
>

-- 
-


Follow IndiaMART.com  for latest updates on this 
and more:  
  Mobile 
Channel: 

Re: Too many documents Exception

[Chris Hostetter]

As noted by others: you should definitely look into sharding your index -- 
fundementally there is no way to have that many documents in a single 
Lucene index.

However: this is a terrible error for you to get, something in the stack 
should have really given you an error when you tried to add the too many 
documents, not latter when you opened the searcher.  I've opened an issue 
to look into adding that...
  https://issues.apache.org/jira/browse/SOLR-6065


Off the top of my head, i don't know if/how you can easily "fix" your 
current index to be usable again.


: Date: Wed, 7 May 2014 09:54:54 +0900
: From: "[Tech Fun]山崎" 
: Reply-To: solr-user@lucene.apache.org
: To: solr-user@lucene.apache.org
: Subject: Too many documents Exception
: 
: Hello everybody,
: 
: Solr 4.3.1(and 4.7.1), Num Docs + Deleted Docs >
: 2147483647(Integer.MAX_VALUE) over
: Caused by: java.lang.IllegalArgumentException: Too many documents,
: composite IndexReaders cannot exceed 2147483647
: 
: It seems to be trouble similar to the unresolved e-mail.
: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201307.mbox/browser
: 
: If How can I fix this?
: This Solr Specification?
: 
: 
: log.
: 
: ERROR org.apache.solr.core.CoreContainer  – Unable to create core: collection1
: org.apache.solr.common.SolrException: Error opening new searcher
: at org.apache.solr.core.SolrCore.(SolrCore.java:821)
: at org.apache.solr.core.SolrCore.(SolrCore.java:618)
: at 
org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
: at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
: at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
: at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
: at java.util.concurrent.FutureTask.run(FutureTask.java:138)
: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
: at java.util.concurrent.FutureTask.run(FutureTask.java:138)
: at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
: at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
: at java.lang.Thread.run(Thread.java:662)
: Caused by: org.apache.solr.common.SolrException: Error opening new searcher
: at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1438)
: at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1550)
: at org.apache.solr.core.SolrCore.(SolrCore.java:796)
: ... 13 more
: Caused by: org.apache.solr.common.SolrException: Error opening Reader
: at 
org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172)
: at 
org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:183)
: at 
org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:179)
: at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1414)
: ... 15 more
: Caused by: java.lang.IllegalArgumentException: Too many documents,
: composite IndexReaders cannot exceed 2147483647
: at 
org.apache.lucene.index.BaseCompositeReader.(BaseCompositeReader.java:77)
: at 
org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:368)
: at 
org.apache.lucene.index.StandardDirectoryReader.(StandardDirectoryReader.java:42)
: at 
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:71)
: at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
: at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
: at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)
: at 
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
: at 
org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169)
: ... 18 more
: ERROR org.apache.solr.core.CoreContainer  –
: null:org.apache.solr.common.SolrException: Unable to create core:
: collection1
: at 
org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1450)
: at org.apache.solr.core.CoreContainer.create(CoreContainer.java:993)
: at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
: at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
: at java.util.concurrent.FutureTask.run(FutureTask.java:138)
: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
: at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
: at java.util.concurrent.FutureTask.run(FutureTask.java:138)
: at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
: at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPool

Re: URLDataSource : indexing from other Solr servers

[Shawn Heisey]

On 5/12/2014 10:11 AM, helder.sepulveda wrote:
> I been trying to index data from other solr servers but the import always
> shows:
> Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.
> Requests: 1, Fetched: 0, Skipped: 0, Processed

I'm wondering why you're using the XPathEntityProcessor instead of
SolrEntityProcessor.

http://wiki.apache.org/solr/DataImportHandler#SolrEntityProcessor

Solr (as of version 3.6) comes with the capability to fully understand
the output from another Solr server, so you should probably be using
that instead of trying to parse XML.

Thanks,
Shawn

Re: Replica active during warming

[Mark Miller]

If you are sure about this, can you file a JIRA issue?
-- 
Mark Miller
about.me/markrmiller

On May 12, 2014 at 8:50:42 PM, lboutros (boutr...@gmail.com) wrote:

Dear All,  

we just finished the migration of a cluster from Solr 4.3.1 to Solr 4.6.1.  
With solr 4.3.1 a node was not considered as active before the end of the  
warming process.  

Now, with solr 4.6.1 a replica is considered as active during the warming  
process.  
This means that if you restart a replica or create a new one, queries will  
be send to this replica and the query will hang until the end of the warming  
process (We do not use cold searchers).  

We have quite long warming queries and this is a big issue.  
Is there a parameter I do not know that could control this behavior ?  

thanks,  

Ludovic.  



-  
Jouve  
France.  
--  
View this message in context: 
http://lucene.472066.n3.nabble.com/Replica-active-during-warming-tp4135274.html 
 
Sent from the Solr - User mailing list archive at Nabble.com.  

Re: Too many documents Exception

[Jack Krupansky]

One of the hard-core Lucene guys is going to have to help you out. Or you 
may have to write some custom code to fix the index for any such shard. If 
you have deleted any documents, it may be sufficient to simply optimize the 
index.


-- Jack Krupansky

-Original Message- 
From: yamazaki

Sent: Wednesday, May 7, 2014 8:15 PM
To: solr-user@lucene.apache.org
Subject: Re: Too many documents Exception

Tanks, Jack.

Is there a way to suppress setting this exception?

For example,
2147483647 ?


When this exception occurs, Index will not be read.
If solrcloud  is used, some data not read.

shard1 documents 2^31-1 over
shard2 documents 2^31-1 not over

shard1 down. shard1 index is dead.

-- yamazaki


2014-05-07 11:01 GMT+09:00 Jack Krupansky :
Lucene only supports 2^31-1 documents in an index, so Solr can only 
support

2^31-1 documents in a single shard.

I think it's a bug that Lucene doesn't throw an exception when more than
that number of documents have been inserted. Instead, you get this error
when Solr tries to read such an overstuffed index.

-- Jack Krupansky

-Original Message- From: [Tech Fun]山崎
Sent: Tuesday, May 6, 2014 8:54 PM
To: solr-user@lucene.apache.org
Subject: Too many documents Exception


Hello everybody,

Solr 4.3.1(and 4.7.1), Num Docs + Deleted Docs >
2147483647(Integer.MAX_VALUE) over
Caused by: java.lang.IllegalArgumentException: Too many documents,
composite IndexReaders cannot exceed 2147483647

It seems to be trouble similar to the unresolved e-mail.
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201307.mbox/browser

If How can I fix this?
This Solr Specification?


log.

ERROR org.apache.solr.core.CoreContainer  – Unable to create core:
collection1
org.apache.solr.common.SolrException: Error opening new searcher
   at org.apache.solr.core.SolrCore.(SolrCore.java:821)
   at org.apache.solr.core.SolrCore.(SolrCore.java:618)
   at
org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
   at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
   at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
   at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.common.SolrException: Error opening new 
searcher

   at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1438)
   at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1550)
   at org.apache.solr.core.SolrCore.(SolrCore.java:796)
   ... 13 more
Caused by: org.apache.solr.common.SolrException: Error opening Reader
   at
org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172)
   at
org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:183)
   at
org.apache.solr.search.SolrIndexSearcher.(SolrIndexSearcher.java:179)
   at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1414)
   ... 15 more
Caused by: java.lang.IllegalArgumentException: Too many documents,
composite IndexReaders cannot exceed 2147483647
   at
org.apache.lucene.index.BaseCompositeReader.(BaseCompositeReader.java:77)
   at
org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:368)
   at
org.apache.lucene.index.StandardDirectoryReader.(StandardDirectoryReader.java:42)
   at
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:71)
   at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
   at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
   at 
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)

   at
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
   at
org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169)
   ... 18 more
ERROR org.apache.solr.core.CoreContainer  –
null:org.apache.solr.common.SolrException: Unable to create core:
collection1
   at
org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1450)
   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:993)
   at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
   at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
   at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:43

Re: Join in solr to get data from two cores

[Walter Underwood]

Probably because we answered a nearly identical request yesterday. It had items 
in one core and counts in a different. Please read all the responses to this 
e-mail.

http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201405.mbox/browser

Specifically, these responses:

http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201405.mbox/browser
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201405.mbox/browser
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201405.mbox/browser

wunder

On May 13, 2014, at 12:03 AM, Kamal Kishore  wrote:

> NO reply from anybody..seems strange ?
> 
> 
> On Fri, May 9, 2014 at 9:47 AM, Kamal Kishore
> wrote:
> 
>> Any updates guys ?
>> 
>> 
>> On Thu, May 8, 2014 at 2:05 PM, Kamal Kishore >> wrote:
>> 
>>> Dear Team,
>>> 
>>> I have two solr cores. One containing products information and second has
>>> customers points. I am looking at solr join to query on first product core
>>> & boost the results based on customer points in second core. I am not able
>>> to frame solr query for this.
>>> 
>>> Moreover, solr is not allowing to get data from both the core.
>>> 
>>> 
>>> With RegardsK
>>> 
>>> Kamal Kishore
>>> 
>>> 
>>> 
>>> 
>> 
> 
> -- 
> -
> 
> 
> Follow IndiaMART.com  for latest updates on this 
> and more:  
>   Mobile 
> Channel: 
> 
> 
> 
> 
> 
> 
> 
> 

--
Walter Underwood
wun...@wunderwood.org

Independent/Selfcontained Solr Unit testing with JUnit

[Vijay Balakrishnan]

Hi,

Is there any way to run self-contained JUnit tests for say a Solr dependent
class where it doesn't depend on Solr being up and running at
localhost:8983 ? I have a collection etc. setup on the Solr server.

Is it possible to mockit with an EmbeddedSolr easily  with a @Before or
@BeforeClass annotation in JUnit4 ?

Any pointers to examples would be awesome(I am also trying to look in the
source).

 TIA,

Vijay

Grouping on int field in SolrCloud raises exception

[Cario, Elaine]

Wondering if anyone else has this issue?

We have a grouping field which we defined as an integer; when we run a query 
grouping on that field it works fine in a non-cloud configuration, but when we 
try the same query in a SolrCloud configuration with multiple shards, we get 
the following error:

Type mismatch: wkcluster was indexed as NUMERIC

Schema:



Query:

q=*:*&group=true&group.field=wkcluster&group.limit=3

We worked around it by redefining the field as string and re-indexing the 
content.  Is there any restriction on the type of a field used for 
grouping/collapsing, especially in a distributed config?

Re: DataImport using SqlEntityProcessor running Out of Memory

[Mikhail Khludnev]

Hello O,
It seems to me (but it's better to look at the heap histogram) that
buffering sub-entities in SortedMapBackedCache blows heap off.
I'm aware about two directions:
- use file based cache instead. I don't know exactly how it works, you can
start from https://issues.apache.org/jira/browse/SOLR-2382 and check how to
enable berkleyDB cache;
- personally, I'm promoting merging resultsets ordered by RDBMS
https://issues.apache.org/jira/browse/SOLR-4799




On Fri, May 9, 2014 at 7:16 PM, O. Olson  wrote:

> I have a Data Schema which is Hierarchical i.e. I have an Entity and a
> number
> of attributes. For a small subset of the Data - about 300 MB, I can do the
> import with 3 GB memory. Now with the entire 4 GB Dataset, I find I cannot
> do the import with 9 GB of memory.
> I am using the SqlEntityProcessor as below:
>
> 
> 
> url="jdbc:sqlserver://localhost\MSSQLSERVER;databaseName=SolrDB;user=solrusr;password=solrusr;"/>
> 
> 
> 
> 
>
>  query="SELECT AttributeValue, EntID FROM ATTR_TABLE
> WHERE AttributeID=1"
> cacheKey="EntID"
> cacheLookup="Entity.EntID"
> processor="SqlEntityProcessor" cacheImpl="SortedMapBackedCache">
>  name="EntityAttribute1" />
> 
>  query="SELECT AttributeValue, EntID FROM ATTR_TABLE
> WHERE AttributeID=2"
> cacheKey="EntID"
> cacheLookup="Entity.EntID"
> processor="SqlEntityProcessor" cacheImpl="SortedMapBackedCache">
>  name="EntityAttribute2" />
> 
>
>
>
> 
> 
> 
>
>
>
> What is the best way to import this data? Doing it without a cache, results
> in many SQL queries. With the cache, I run out of memory.
>
> I’m curious why 4GB of data cannot entirely fit in memory. One thing I need
> to mention is that I have about 400 to 500 attributes.
>
> Thanks in advance for any helpful advice.
> O. O.
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/DataImport-using-SqlEntityProcessor-running-Out-of-Memory-tp4135080.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


 

Re: retreive all the fields in join

[Aman Tandon]

Thanks Walter yeah we are now trying to use the external file field as
mentioned by you.
On May 12, 2014 11:11 PM, "Walter Underwood"  wrote:

> Top management has given requirements that force a broken design. They are
> requiring something that is impossible with Solr.
>
> 1. Flatten the data. You get one table, no joins.
>
> 2. 12M records is not a big Solr index. That should work fine.
>
> 3. If the supplier activity points are updated frequently, you could use
> an external file field for those, but they still need to be flattened.
>
> wunder
>
> On May 12, 2014, at 7:21 AM, Aman Tandon  wrote:
>
> > Yeah i understand but i got the requirement from the top management,
> > requirements are:
> > core1:  in this we want to keep the supplier activity points
> > case 2: we want to boost those records which are present in core1 by the
> > amount of supplier activity points.
> >
> > I know we can keep that supplier score in same core but this requires the
> > full indexing of 12M records and suppliers are of about 1lacs which won't
> > cost much.
> >
> > With Regards
> > Aman
> >
> >
> > On Mon, May 12, 2014 at 7:44 PM, Erick Erickson  >wrote:
> >
> >> Any time you find yourself trying to use Solr like a DB, stop.
> >>
> >> Solr joins are _not_ DB joins, the data from the "from" core is not
> >> returned (I think there are a few special cases where you can make
> >> this happen though).
> >>
> >> Try denormalizing your data if at all possible, that's what Solr docs
> >> best... search single records.
> >>
> >> Best,
> >> Erick
> >>
> >> On Sun, May 11, 2014 at 6:40 PM, Aman Tandon 
> >> wrote:
> >>> please help me out here!!
> >>>
> >>> With Regards
> >>> Aman Tandon
> >>>
> >>>
> >>> On Sun, May 11, 2014 at 1:44 PM, Aman Tandon  >>> wrote:
> >>>
>  Hi,
> 
>  Is there a way possible to retrieve all the fields present in both the
>  cores(core 1 and core2).
> 
>  e.g.
>  core1: {id:111,name: "abc" }
> 
>  core2: {page:17, type: "fiction"}
> 
>  I want is that, on querying both the cores I want to retrieve the
> >> results
>  containing all the 4 fields, fields id, name from core1 and page, type
> >> from
>  core2. Is it possible?
> 
>  With Regards
>  Aman Tandon
> 
> >>
>
> --
> Walter Underwood
> wun...@wunderwood.org
>
>
>
>

Re: solr optimize on fnm file

[Erick Erickson]

Oh my! I see what your problem is, but I rather doubt that it'll be
addressed. You've obviously been stress-testing the indexing process
and have a bunch of garbage left over that's not getting removed on
optimize. But it's such a unique case that I don't know if anyone is
really interested in fixing it.

I'll bring it up on the dev list, but I expect you'll need to blow
away your entire index and re-index from scratch.

Best,
Erick

On Tue, May 6, 2014 at 4:09 PM, googoo  wrote:
> For our setup, the file size is 123M. Internal it has 2.6M fields.
>
> The problem is facet operation. It take a while for facet.
> we are stuck in below call stack for 11 second.
>
> java.util.HashMap.transfer(Unknown Source)
> java.util.HashMap.resize(Unknown Source)
> java.util.HashMap.addEntry(Unknown Source)
> java.util.HashMap.put(Unknown Source)
> org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:285)
> org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:302)
> org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:251)
> org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:276)
> org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:220)
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1116)
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1106)
> org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:574)
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:429)
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/solr-optimize-on-fnm-file-tp4134969p4135037.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: query(subquery, default) filters results

[Aman Tandon]

Thanks Yonik and Chris for the the brief explanation. Me too was unknown
about the behavior of query parser.
Yonik can you help that how can i learn deeply about the query parsers and
how to gain expertise in debug query.



With Regards
Aman Tandon


On Wed, May 7, 2014 at 12:35 AM, Yonik Seeley  wrote:

> On Tue, May 6, 2014 at 5:08 AM, Matteo Grolla 
> wrote:
> > Hi everybody,
> > I'm having troubles with the function query
> >
> > "query(subquery, default)"
> http://wiki.apache.org/solr/FunctionQuery#query
> >
> > running this
> >
> > http://localhost:8983/solr/select?q=query($qq,1)&qq={!dismaxqf=text}hard 
> > drive
>
> The default query syntax is lucene, so "query(..." will just be parsed as
> text.
> Try q={!func}query($qq,1)
> OR
> defType=func&q=query($qq,1)
>
> -Yonik
> http://heliosearch.org - facet functions, subfacets, off-heap filters
> + fieldcache
>

getting direct link to solr result.

[blach]

Hello all!

I have been using solr for a few days, but I still don't understand, how can
I get direct link to open the document i'm looking for.

I tried to do that, but the only information I can retrieve from the Json
result from Solr is ID, Name, Modified date ... 

well, I'm working on android application, and I want to make the user get a
direct link to the file he searched for.

thanks.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/getting-direct-link-to-solr-result-tp4135084.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solrj Default Data Format

[Michael Della Bitta]

Hi Furkan,

If I were to guess, the XML format is more cross-compatible with different
versions of SolrJ. But it might not be intentional.

In any case, feeding your SolrServer a BinaryResponseParser will switch it
over to javabin.

Michael Della Bitta

Applications Developer

o: +1 646 532 3062

appinions inc.

“The Science of Influence Marketing”

18 East 41st Street

New York, NY 10017

t: @appinions  | g+:
plus.google.com/appinions
w: appinions.com 


On Thu, May 8, 2014 at 10:17 AM, Furkan KAMACI wrote:

> Hi;
>
> I found the reason of weird format at my previous mail. Now I capture the
> data with wireshark and I see that it is pure XML and content type is set
> to application/xml?
>
> Any ideas about why it is not javabin?
>
> Thanks;
> Furkan KAMACI
>
>
> 2014-05-07 22:16 GMT+03:00 Furkan KAMACI :
>
> > Hmmm, I see that it is like XML format but not. I have added three
> > documents but has something like that:
> >
> > 
> > 
> > id1
> > id2
> > id3
> > id4
> > d1
> > d2
> > d3
> > d4
> > 
> > 
> > 
> > 
> > 
> >
> > is this javabin format? I mean optimizing XML and having a first byte of
> > "2"?
> >
> > Thanks;
> > Furkan KAMACI
> >
> >
> > 2014-05-07 22:04 GMT+03:00 Furkan KAMACI :
> >
> > Hi;
> >>
> >> I am testing Solrj. I use Solr 4.5.1 and HttpSolrServer for my test. I
> >> just generate some SolrInputDocuments and call add method of server to
> add
> >> them. When  I track the request I see that data is at XML format
> instead of
> >> javabin. Do I miss anything?
> >>
> >> Thanks;
> >> Furkan KAMACI
> >>
> >
> >
>

KeywordTokenizerFactory splits the string for the exclamation mark

[nativecoder]

Hi jack

Please have a look at this


I have a following field settings in solr schema









As you can see Exact_Word has the KeywordTokenizerFactory and that should
treat the string as it is.

Following is my responseHeader. As you can see I am searching my string
only in the filed Exact_Word and expecting it to return the Word field and
the score

"responseHeader":{
"status":0,
"QTime":14,
"params":{
  "explainOther":"",
  "fl":"Word,score",
  "debugQuery":"on",
  "indent":"on",
  "start":"0",
  "q":"d!sdasdsdwasd!a...@dsadsadas.edu",
  "qf":"Exact_Word",
  "wt":"json",
  "fq":"",
  "version":"2.2",
  "rows":"10"}},


But when I enter email with the following string "d!
sdasdsdwasd...@dsadsadas.edu" it splits the string to two. I was under the
impression that KeywordTokenizerFactory will treat the string as it is.

Following is the query debug result. There you can see it has split the word
 "parsedquery":"+((DisjunctionMaxQuery((Exact_Word:d))
-DisjunctionMaxQuery((Exact_Word:sdasdsdwasd...@dsadsadas.edu)))~1)",

can someone please tell why it produce the query result as this

If I put a string without the "!" sign as below, the produced query will be
as below
 "parsedquery":"+DisjunctionMaxQuery((
Exact_Word:d_sdasdsdwasd_...@dsadsadas.edu))",. This is what I expected
solr to even with the "!" mark. with "_" mark it wont do a string split and
treats the string as it is

I thought if the KeywordTokenizerFactory is applied then it should return
the exact string as it is

Please help me to understand what is going wrong here




--
View this message in context: 
http://lucene.472066.n3.nabble.com/KeywordTokenizerFactory-splits-the-string-for-the-exclamation-mark-tp4135474.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Independent/Selfcontained Solr Unit testing with JUnit

[Shawn Heisey]

On 5/13/2014 12:46 PM, Vijay Balakrishnan wrote:
> Is there any way to run self-contained JUnit tests for say a Solr dependent
> class where it doesn't depend on Solr being up and running at
> localhost:8983 ? I have a collection etc. setup on the Solr server.
> 
> Is it possible to mockit with an EmbeddedSolr easily  with a @Before or
> @BeforeClass annotation in JUnit4 ?
> 
> Any pointers to examples would be awesome(I am also trying to look in the
> source).

An example of a Solr unit test that fires up Jetty (actually, more than
one instance of Jetty) before testing is located here in the source
download or checkout:

solr/solrj/src/test/org/apache/solr/client/solrj/TestLBHttpSolrServer.java

Thanks,
Shawn

RE: Easises way to insatll solr cloud with tomcat

[Matt Kuiper (Springblox)]

Check out http://heliosearch.com/download.html 

It is a distribution of Apache Solr packaged with Tomcat.

I have found it simple to use.

Matt

-Original Message-
From: Aman Tandon [mailto:amantandon...@gmail.com] 
Sent: Monday, May 12, 2014 6:24 AM
To: solr-user@lucene.apache.org
Subject: Re: Easises way to insatll solr cloud with tomcat

Can anybody help me out??

With Regards
Aman Tandon


On Mon, May 12, 2014 at 1:24 PM, Aman Tandon wrote:

> Hi,
>
> I tried to set up solr cloud with jetty which works fine. But in our 
> production environment we uses tomcat so i need to set up the solr 
> cloud with the tomcat. So please help me out to how to setup solr 
> cloud with tomcat on single machine.
>
> Thanks in advance.
>
> With Regards
> Aman Tandon
>

search multiple cores

[Jay Potharaju]

Hi,
I am trying to join across multiple cores using query time join. Following
is my setup
3 cores - Solr 4.7
core1:  0.5 million documents
core2: 4 million documents and growing. This contains the child documents
for documents in core1.
core3: 2 million documents and growing. Contains records from all users.

 core2 contains documents that are accessible to each user based on their
permissions. The number of documents accessible to a user range from couple
of 1000s to 100,000.

I would like to get results by combining all three cores. For each search I
get documents from core3 and then query core1 to get parent documents &
then core2 to get the appropriate child documents depending of user
permissions.

I 'm referring to this link to join across cores
http://stackoverflow.com/questions/12665797/is-solr-4-0-capable-of-using-join-for-multiple-core

{!join from=fromField to=toField fromIndex=fromCoreName}fromQuery

This is not working for me. Can anyone suggest why it is not working. Any
pointers on how to search across multiple cores.

thanks



J

Re: What is the usage of solr.NumericPayloadTokenFilterFactory

[Jack Krupansky]

I do have basic coverage for that filter (and all other filters) and the 
parameter values in my e-book:


http://www.lulu.com/us/en/shop/jack-krupansky/solr-4x-deep-dive-early-access-release-7/ebook/product-21203548.html

That said, are you sure you want to be using the payload feature of Lucene?

-- Jack Krupansky

-Original Message- 
From: ienjreny

Sent: Monday, May 12, 2014 12:51 PM
To: solr-user@lucene.apache.org
Subject: What is the usage of solr.NumericPayloadTokenFilterFactory

Dears:
Can any body explain at easy way what is the benefits of
solr.NumericPayloadTokenFilterFactory and what is acceptable values for
typeMatch

Thanks in advance



--
View this message in context: 
http://lucene.472066.n3.nabble.com/What-is-the-usage-of-solr-NumericPayloadTokenFilterFactory-tp4135326.html
Sent from the Solr - User mailing list archive at Nabble.com. 

Error when creating collection

[Mark Olsen]

Solr version: 4.2.1 

I'm creating a collection via Java using this function call: 

String collection = "profile-2"; 
CoreAdminRequest.Create createRequest = new CoreAdminRequest.Create(); 
createRequest.setCoreName(collection); 
createRequest.setCollection(collection); 
createRequest.setInstanceDir(collection); 
createRequest.setNumShards(1); 
createRequest.process(server); 

It is timing out with this exception (from the solr.out logs): 

SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore 
'profile-2': Could not get shard_id for core: profile-2 
coreNodeName:192.168.1.152:8983_solr_profile-2 
at 
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:483)
 
at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:140)
 
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
 
at 
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:591)
 
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192)
 
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
 
... 
Caused by: org.apache.solr.common.SolrException: Could not get shard_id for 
core: profile-2 coreNodeName:192.168.1.152:8983_solr_profile-2 
at 
org.apache.solr.cloud.ZkController.doGetShardIdProcess(ZkController.java:1221) 
at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1290) 
at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:861) 

In a "development" environment the zookeeper/solr instances are running with 
elevated permissions and this function worked without error. 
In a "test" environment (which matches the "production" environment) the 
permissions are more restricted. I made sure the group/owner of the 
/usr/local/solr directory are set up to be the correct user. 


Any insight into potential file permissions that I may be overlooking? 


Thank you, 
Mark 

Re: Join in solr to get data from two cores

[Erick Erickson]

You really have to provide more detail here.

bq: Moreover, solr is not allowing to get data from both the core.

What do you mean? the second core is unavailable? Solr joins
do not return data from the  "from" table. I really suggest you
try denormalizing the data first, don't try to use Solr like
you would SQL, only resort to joins etc. once you've exhausted
your creativity as far as denormalizing data is concerned.

Best,
Erick

On Thu, May 8, 2014 at 1:35 AM, Kamal Kishore
 wrote:
> Dear Team,
>
> I have two solr cores. One containing products information and second has
> customers points. I am looking at solr join to query on first product core
> & boost the results based on customer points in second core. I am not able
> to frame solr query for this.
>
> Moreover, solr is not allowing to get data from both the core.
>
>
> With RegardsK
>
> Kamal Kishore
>
> --
> -
> 
>
> Follow IndiaMART.com  for latest updates on this
> and more: 
>   Mobile 
> Channel:
> 
>
>
>
>
>
>
>