Searching inside parent AND children, returning only parents.

[Marre]

I have been experimenting with solr for a couple of weeks but i've been stuck
on a query i would like to execute for a couple of days now.

I have a nested data structure where I'm using a fq like this:

{!parent which="parentDoc:true"}parentDoc:false AND 

This matches my child documents and returns the parent to those children. I
am very pleased with that. BUT the problem i have is if there is a match
directly inside the parent and nothing in the children. I will not get a
response.

I would make it so that in some way there is a OR condition of some sort
making it so, any document can match the searched term AND parentDoc:false
OR the above filter query.

Is this even possible to execute within one query in solr, or do i have to
make two? I have not found any information about this issue, making me to
believe im just missing something trivial.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Searching-inside-parent-AND-children-returning-only-parents-tp4285078.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: error rendering solr spatial in geoserver

[Ere Maijala]

It would have been _really_ nice if this had been in the release notes. 
Made me also scratch my head for a while when upgrading to Solr 6. 
Additionally, this makes a rolling upgrade from Solr 5.x a bit more 
scary since you have to update the collection schema to make the Solr 6 
nodes work while making sure that no Solr 5 node reloads the configuration.


--Ere

30.6.2016, 3.46, David Smiley kirjoitti:

For polygons in 6.0 you need to set
spatialContextFactory="org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory"
-- see
https://cwiki.apache.org/confluence/display/solr/Spatial+Search and the
example.  And of course as you probably already know, put the JTS jar on
Solr's classpath.  What likely tripped you up between 5x and 6x is the
change in value of the spatialContextFactory as a result in organizational
package moving "com.spatial4j.core" to "org.locationtech.spatial4j".

On Wed, Jun 29, 2016 at 12:44 PM tkg_cangkul  wrote:


hi erick, thx for your reply.

i've solve this problem.
i got this error when i use solr 6.0.0
so i try to downgrade my solr to version 5.5.0 and it's successfull


On 29/06/16 22:39, Erick Erickson wrote:

There is not nearly enough information here to say anything very helpful.
What does your schema look like for this field?
What does the input look like?
How are you pulling data from geoserver?

You might want to review:
http://wiki.apache.org/solr/UsingMailingLists

Best,
Erick

On Wed, Jun 29, 2016 at 2:31 AM, tkg_cangkul mailto:yuza.ras...@gmail.com>> wrote:

hi, i try to load data spatial from solr with geoserver.
when i try to show the layer preview i've got this error message.

error


anybody can help me pls?




--

Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com



--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

Re: error rendering solr spatial in geoserver

[Youzha]

well i didn`t know if the package was moved to "
org.locationtech.spatial4j" in 6.x. i will try this as your suggestion. thx
for your help anyway.
On Jun 30, 2016 8:46 AM, "David Smiley"  wrote:

> For polygons in 6.0 you need to set
>
> spatialContextFactory="org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory"
> -- see
> https://cwiki.apache.org/confluence/display/solr/Spatial+Search and the
> example.  And of course as you probably already know, put the JTS jar on
> Solr's classpath.  What likely tripped you up between 5x and 6x is the
> change in value of the spatialContextFactory as a result in organizational
> package moving "com.spatial4j.core" to "org.locationtech.spatial4j".
>
> On Wed, Jun 29, 2016 at 12:44 PM tkg_cangkul 
> wrote:
>
> > hi erick, thx for your reply.
> >
> > i've solve this problem.
> > i got this error when i use solr 6.0.0
> > so i try to downgrade my solr to version 5.5.0 and it's successfull
> >
> >
> > On 29/06/16 22:39, Erick Erickson wrote:
> > > There is not nearly enough information here to say anything very
> helpful.
> > > What does your schema look like for this field?
> > > What does the input look like?
> > > How are you pulling data from geoserver?
> > >
> > > You might want to review:
> > > http://wiki.apache.org/solr/UsingMailingLists
> > >
> > > Best,
> > > Erick
> > >
> > > On Wed, Jun 29, 2016 at 2:31 AM, tkg_cangkul  > > > wrote:
> > >
> > > hi, i try to load data spatial from solr with geoserver.
> > > when i try to show the layer preview i've got this error message.
> > >
> > > error
> > >
> > >
> > > anybody can help me pls?
> > >
> > >
> >
> > --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>

Re: error rendering solr spatial in geoserver

[Youzha]

i agree for this. :D
On Jun 30, 2016 4:12 PM, "Ere Maijala"  wrote:

> It would have been _really_ nice if this had been in the release notes.
> Made me also scratch my head for a while when upgrading to Solr 6.
> Additionally, this makes a rolling upgrade from Solr 5.x a bit more scary
> since you have to update the collection schema to make the Solr 6 nodes
> work while making sure that no Solr 5 node reloads the configuration.
>
> --Ere
>
> 30.6.2016, 3.46, David Smiley kirjoitti:
>
>> For polygons in 6.0 you need to set
>>
>> spatialContextFactory="org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory"
>> -- see
>> https://cwiki.apache.org/confluence/display/solr/Spatial+Search and the
>> example.  And of course as you probably already know, put the JTS jar on
>> Solr's classpath.  What likely tripped you up between 5x and 6x is the
>> change in value of the spatialContextFactory as a result in organizational
>> package moving "com.spatial4j.core" to "org.locationtech.spatial4j".
>>
>> On Wed, Jun 29, 2016 at 12:44 PM tkg_cangkul 
>> wrote:
>>
>> hi erick, thx for your reply.
>>>
>>> i've solve this problem.
>>> i got this error when i use solr 6.0.0
>>> so i try to downgrade my solr to version 5.5.0 and it's successfull
>>>
>>>
>>> On 29/06/16 22:39, Erick Erickson wrote:
>>>
 There is not nearly enough information here to say anything very
 helpful.
 What does your schema look like for this field?
 What does the input look like?
 How are you pulling data from geoserver?

 You might want to review:
 http://wiki.apache.org/solr/UsingMailingLists

 Best,
 Erick

 On Wed, Jun 29, 2016 at 2:31 AM, tkg_cangkul >>> > wrote:

 hi, i try to load data spatial from solr with geoserver.
 when i try to show the layer preview i've got this error message.

 error


 anybody can help me pls?



>>> --
>>>
>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>> http://www.solrenterprisesearchserver.com
>>
>>
> --
> Ere Maijala
> Kansalliskirjasto / The National Library of Finland
>

Solr node crashes while indexing - Too many open files

[Mads Tomasgård Bjørgan]

Hello,
We're indexing a large set of files using Solr 6.1.0, running a SolrCloud by 
utilizing ZooKeeper 3.4.8.

We have two ensembles - and both clusters are running on three of their own 
respective VMs (CentOS 7). We first thought the error was due to CDCR - as we 
were trying to index a large amount of documents which had to be replicated to 
the target cluster. However, we got the same error even after turning of CDCR - 
which indicates CDCR wasn't the problem after all.

After indexing between 20 000 to 35 000 documents to the source cluster does 
the File Descriptor Count reach 4096 for one of the solr-nodes - and the 
respective node crashes. The count grows quite linearly as time goes. The 
remaining 2 nodes in the cluster is not affected at all, and their logs had no 
relevant posts.  We found the following errors for the crashing node in its log:

2016-06-30 08:23:12.459 ERROR 
(updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
 x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 c:DIPS) 
[c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
o.a.s.u.StreamingSolrClients error
java.net.SocketException: Too many open files
(...)
2016-06-30 08:23:12.460 ERROR 
(updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
 x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 c:DIPS) 
[c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
o.a.s.u.StreamingSolrClients error
java.net.SocketException: Too many open files
(...)
2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
x:DIPS_shard1_replica1] o.a.s.h.RequestHandlerBase 
org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
 2 Async exceptions during distributed update:
Too many open files
Too many open files
(...)
2016-06-30 08:23:12.461 INFO  (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
x:DIPS_shard1_replica1] o.a.s.c.S.Request [DIPS_shard1_replica1]  webapp=/solr 
path=/update params={version=2.2} status=-1 QTime=5
2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall 
null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
 2 Async exceptions during distributed update:
Too many open files
Too many open files
()

2016-06-30 08:23:12.461 WARN  (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall invalid return code: -1
2016-06-30 08:23:38.108 INFO  (qtp314337396-20) [c:DIPS s:shard1 r:core_node1 
x:DIPS_shard1_replica1] o.a.s.c.S.Request [DIPS_shard1_replica1]  webapp=/solr 
path=/select 
params={df=_text_&distrib=false&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://10.0.106.115:443/solr/DIPS_shard1_replica1/&rows=10&version=2&q=*:*&NOW=1467275018057&isShard=true&wt=javabin&_=1467275017220}
 hits=30218 status=0 QTime=1

Running netstat -n -p on the VM that yields the exceptions reveals that there 
is at least 1 800 TCP connections (not counted how many - the netstat command 
filled the entire PuTTY window yielding 2 000 lines) waiting to be closed:
tcp6  70  0 10.0.106.115:34531  10.0.106.114:443CLOSE_WAIT  
21658/java
We're running the SolrCloud on 443, and the IP's belong to the VMs. We also 
tried adjusting the ulimit for the machine to 100 000 - without any results..

Greetings,
Mads

RE: Solr node crashes while indexing - Too many open files

[Markus Jelsma]

Mads, some distributions require different steps for increasing max_open_files. 
Check how it works vor CentOS specifically.

Markus

 
 
-Original message-
> From:Mads Tomasgård Bjørgan 
> Sent: Thursday 30th June 2016 10:52
> To: solr-user@lucene.apache.org
> Subject: Solr node crashes while indexing - Too many open files
> 
> Hello,
> We're indexing a large set of files using Solr 6.1.0, running a SolrCloud by 
> utilizing ZooKeeper 3.4.8.
> 
> We have two ensembles - and both clusters are running on three of their own 
> respective VMs (CentOS 7). We first thought the error was due to CDCR - as we 
> were trying to index a large amount of documents which had to be replicated 
> to the target cluster. However, we got the same error even after turning of 
> CDCR - which indicates CDCR wasn't the problem after all.
> 
> After indexing between 20 000 to 35 000 documents to the source cluster does 
> the File Descriptor Count reach 4096 for one of the solr-nodes - and the 
> respective node crashes. The count grows quite linearly as time goes. The 
> remaining 2 nodes in the cluster is not affected at all, and their logs had 
> no relevant posts.  We found the following errors for the crashing node in 
> its log:
> 
> 2016-06-30 08:23:12.459 ERROR 
> (updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
>  x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 c:DIPS) 
> [c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
> o.a.s.u.StreamingSolrClients error
> java.net.SocketException: Too many open files
> (...)
> 2016-06-30 08:23:12.460 ERROR 
> (updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
>  x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 c:DIPS) 
> [c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
> o.a.s.u.StreamingSolrClients error
> java.net.SocketException: Too many open files
> (...)
> 2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.h.RequestHandlerBase 
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>  2 Async exceptions during distributed update:
> Too many open files
> Too many open files
> (...)
> 2016-06-30 08:23:12.461 INFO  (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.c.S.Request [DIPS_shard1_replica1]  
> webapp=/solr path=/update params={version=2.2} status=-1 QTime=5
> 2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall 
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>  2 Async exceptions during distributed update:
> Too many open files
> Too many open files
> ()
> 
> 2016-06-30 08:23:12.461 WARN  (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall invalid return code: -1
> 2016-06-30 08:23:38.108 INFO  (qtp314337396-20) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.c.S.Request [DIPS_shard1_replica1]  
> webapp=/solr path=/select 
> params={df=_text_&distrib=false&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://10.0.106.115:443/solr/DIPS_shard1_replica1/&rows=10&version=2&q=*:*&NOW=1467275018057&isShard=true&wt=javabin&_=1467275017220}
>  hits=30218 status=0 QTime=1
> 
> Running netstat -n -p on the VM that yields the exceptions reveals that there 
> is at least 1 800 TCP connections (not counted how many - the netstat command 
> filled the entire PuTTY window yielding 2 000 lines) waiting to be closed:
> tcp6  70  0 10.0.106.115:34531  10.0.106.114:443
> CLOSE_WAIT  21658/java
> We're running the SolrCloud on 443, and the IP's belong to the VMs. We also 
> tried adjusting the ulimit for the machine to 100 000 - without any results..
> 
> Greetings,
> Mads
> 

RE: Solr node crashes while indexing - Too many open files

[Mads Tomasgård Bjørgan]

That's true, but I was hoping there would be another way to solve this issue as 
it's not considered preferable in our situation.

Is it normal behavior for Solr to open over 4000 files without closing them 
properly? Is it for example possible to adjust autoCommit-settings I 
solrconfig.xml for forcing Solr to close the files?

Any help is appreciated :-)

-Original Message-
From: Markus Jelsma [mailto:markus.jel...@openindex.io] 
Sent: torsdag 30. juni 2016 11.41
To: solr-user@lucene.apache.org
Subject: RE: Solr node crashes while indexing - Too many open files

Mads, some distributions require different steps for increasing max_open_files. 
Check how it works vor CentOS specifically.

Markus

 
 
-Original message-
> From:Mads Tomasgård Bjørgan 
> Sent: Thursday 30th June 2016 10:52
> To: solr-user@lucene.apache.org
> Subject: Solr node crashes while indexing - Too many open files
> 
> Hello,
> We're indexing a large set of files using Solr 6.1.0, running a SolrCloud by 
> utilizing ZooKeeper 3.4.8.
> 
> We have two ensembles - and both clusters are running on three of their own 
> respective VMs (CentOS 7). We first thought the error was due to CDCR - as we 
> were trying to index a large amount of documents which had to be replicated 
> to the target cluster. However, we got the same error even after turning of 
> CDCR - which indicates CDCR wasn't the problem after all.
> 
> After indexing between 20 000 to 35 000 documents to the source cluster does 
> the File Descriptor Count reach 4096 for one of the solr-nodes - and the 
> respective node crashes. The count grows quite linearly as time goes. The 
> remaining 2 nodes in the cluster is not affected at all, and their logs had 
> no relevant posts.  We found the following errors for the crashing node in 
> its log:
> 
> 2016-06-30 08:23:12.459 ERROR 
> (updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
>  x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 c:DIPS) 
> [c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
> o.a.s.u.StreamingSolrClients error
> java.net.SocketException: Too many open files
> (...)
> 2016-06-30 08:23:12.460 ERROR 
> (updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
>  x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 c:DIPS) 
> [c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
> o.a.s.u.StreamingSolrClients error
> java.net.SocketException: Too many open files
> (...)
> 2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.h.RequestHandlerBase 
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>  2 Async exceptions during distributed update:
> Too many open files
> Too many open files
> (...)
> 2016-06-30 08:23:12.461 INFO  (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.c.S.Request [DIPS_shard1_replica1]  
> webapp=/solr path=/update params={version=2.2} status=-1 QTime=5
> 2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall 
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>  2 Async exceptions during distributed update:
> Too many open files
> Too many open files
> ()
> 
> 2016-06-30 08:23:12.461 WARN  (qtp314337396-18) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall invalid return code: -1
> 2016-06-30 08:23:38.108 INFO  (qtp314337396-20) [c:DIPS s:shard1 r:core_node1 
> x:DIPS_shard1_replica1] o.a.s.c.S.Request [DIPS_shard1_replica1]  
> webapp=/solr path=/select 
> params={df=_text_&distrib=false&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://10.0.106.115:443/solr/DIPS_shard1_replica1/&rows=10&version=2&q=*:*&NOW=1467275018057&isShard=true&wt=javabin&_=1467275017220}
>  hits=30218 status=0 QTime=1
> 
> Running netstat -n -p on the VM that yields the exceptions reveals that there 
> is at least 1 800 TCP connections (not counted how many - the netstat command 
> filled the entire PuTTY window yielding 2 000 lines) waiting to be closed:
> tcp6  70  0 10.0.106.115:34531  10.0.106.114:443
> CLOSE_WAIT  21658/java
> We're running the SolrCloud on 443, and the IP's belong to the VMs. We also 
> tried adjusting the ulimit for the machine to 100 000 - without any results..
> 
> Greetings,
> Mads
> 

RE: Solr node crashes while indexing - Too many open files

[Markus Jelsma]

Yes, that is quite normal for a busy search engine, especially for cloud 
environments. We always start by increasing it to 64k minimum when provisioning 
machines.
Markus
 
-Original message-
> From:Mads Tomasgård Bjørgan 
> Sent: Thursday 30th June 2016 13:05
> To: solr-user@lucene.apache.org
> Subject: RE: Solr node crashes while indexing - Too many open files
> 
> That's true, but I was hoping there would be another way to solve this issue 
> as it's not considered preferable in our situation.
> 
> Is it normal behavior for Solr to open over 4000 files without closing them 
> properly? Is it for example possible to adjust autoCommit-settings I 
> solrconfig.xml for forcing Solr to close the files?
> 
> Any help is appreciated :-)
> 
> -Original Message-
> From: Markus Jelsma [mailto:markus.jel...@openindex.io] 
> Sent: torsdag 30. juni 2016 11.41
> To: solr-user@lucene.apache.org
> Subject: RE: Solr node crashes while indexing - Too many open files
> 
> Mads, some distributions require different steps for increasing 
> max_open_files. Check how it works vor CentOS specifically.
> 
> Markus
> 
>  
>  
> -Original message-
> > From:Mads Tomasgård Bjørgan 
> > Sent: Thursday 30th June 2016 10:52
> > To: solr-user@lucene.apache.org
> > Subject: Solr node crashes while indexing - Too many open files
> > 
> > Hello,
> > We're indexing a large set of files using Solr 6.1.0, running a SolrCloud 
> > by utilizing ZooKeeper 3.4.8.
> > 
> > We have two ensembles - and both clusters are running on three of their own 
> > respective VMs (CentOS 7). We first thought the error was due to CDCR - as 
> > we were trying to index a large amount of documents which had to be 
> > replicated to the target cluster. However, we got the same error even after 
> > turning of CDCR - which indicates CDCR wasn't the problem after all.
> > 
> > After indexing between 20 000 to 35 000 documents to the source cluster 
> > does the File Descriptor Count reach 4096 for one of the solr-nodes - and 
> > the respective node crashes. The count grows quite linearly as time goes. 
> > The remaining 2 nodes in the cluster is not affected at all, and their logs 
> > had no relevant posts.  We found the following errors for the crashing node 
> > in its log:
> > 
> > 2016-06-30 08:23:12.459 ERROR 
> > (updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
> >  x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 
> > c:DIPS) [c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
> > o.a.s.u.StreamingSolrClients error
> > java.net.SocketException: Too many open files
> > (...)
> > 2016-06-30 08:23:12.460 ERROR 
> > (updateExecutor-2-thread-22-processing-https:10.0.106.168:443//solr//DIPS_shard3_replica1
> >  x:DIPS_shard1_replica1 r:core_node1 n:10.0.106.115:443_solr s:shard1 
> > c:DIPS) [c:DIPS s:shard1 r:core_node1 x:DIPS_shard1_replica1] 
> > o.a.s.u.StreamingSolrClients error
> > java.net.SocketException: Too many open files
> > (...)
> > 2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 
> > r:core_node1 x:DIPS_shard1_replica1] o.a.s.h.RequestHandlerBase 
> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >  2 Async exceptions during distributed update:
> > Too many open files
> > Too many open files
> > (...)
> > 2016-06-30 08:23:12.461 INFO  (qtp314337396-18) [c:DIPS s:shard1 
> > r:core_node1 x:DIPS_shard1_replica1] o.a.s.c.S.Request 
> > [DIPS_shard1_replica1]  webapp=/solr path=/update params={version=2.2} 
> > status=-1 QTime=5
> > 2016-06-30 08:23:12.461 ERROR (qtp314337396-18) [c:DIPS s:shard1 
> > r:core_node1 x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall 
> > null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> >  2 Async exceptions during distributed update:
> > Too many open files
> > Too many open files
> > ()
> > 
> > 2016-06-30 08:23:12.461 WARN  (qtp314337396-18) [c:DIPS s:shard1 
> > r:core_node1 x:DIPS_shard1_replica1] o.a.s.s.HttpSolrCall invalid return 
> > code: -1
> > 2016-06-30 08:23:38.108 INFO  (qtp314337396-20) [c:DIPS s:shard1 
> > r:core_node1 x:DIPS_shard1_replica1] o.a.s.c.S.Request 
> > [DIPS_shard1_replica1]  webapp=/solr path=/select 
> > params={df=_text_&distrib=false&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://10.0.106.115:443/solr/DIPS_shard1_replica1/&rows=10&version=2&q=*:*&NOW=1467275018057&isShard=true&wt=javabin&_=1467275017220}
> >  hits=30218 status=0 QTime=1
> > 
> > Running netstat -n -p on the VM that yields the exceptions reveals that 
> > there is at least 1 800 TCP connections (not counted how many - the netstat 
> > command filled the entire PuTTY window yielding 2 000 lines) waiting to be 
> > closed:
> > tcp6  70  0 10.0.106.115:34531  10.0.106.114:443
> > CLOSE_WAIT  21658/java
> > We're

Remove from list

[Colin Hunter]

Hi,

Could you please remove my email from the mailing list (for now).

Many thanks for the help and resource you have provided.
Colin Hunter

-- 
www.gfc.uk.net

How to best serialize/deserialize a SolrInputDocument?

[Sebastian Riemer]

Hi,

I am looking for a way to serialize a SolrInputDocument.

I want to store the serialized document in a MySQL table.

Later I want to deserialize that document and send it to the Solr server.

Currently I am looking at org.apache.solr.client.solrj.request.UpdateRequest 
and JavaBinUpdateRequestCodec. There are two methods, marshal and unmarshal 
which look like I could use for that purpose.

I'd simply create an UpdateRequest, add the document to it, call marshal, save 
the OutputStream somehow in the MySQL table. When retrieving I pass the value 
from the MySQL as InputStream to the unmarshal method, get my UpdateRequest 
object, iterate the contained SolrInputDocument and send it to the server.

Am I on the right track, or is there a better approach?

The background to this is, that we want backup the generated documents which 
are indexed with solr. So if a client restores a backup, that MySQL table with 
the serialized documents can be used to rebuild the index as quickly as 
possible.

Thanks,
Sebastian

RE: Solr 5.3.1 - Synonym is not working as expected

[santoshkumar.turangi]

Hi Team,

Hope you are doing good !!

We are using Solr 5.3.1 version as our search engine. This setup is provided by 
the Bitnami cloud and the amazon AMI is ami-50a47e23.

We have a website which has content in Chinese. We use Nutch crawler to crawl 
the entire website and index it to the Solr collection. We have configured few 
fields including text field with Cinese tokenizers. When user search with 
Chinese characters, we are able to see the relevant results. We wanted to see 
the same results when user types in English or Pinyin characters. So, we have 
included synonym file and respective tokenizer added to the schema.xml file. We 
are not able to get any results after doing these changes. Below is the 
configuration we did in schema.xml. The synonym file is a mapping of Chinese 
word with equivalent English and pinyin words.



  
   
   

   
   
   
  




The output with query debug is providing the below result. The synonym 
configured for the English word is actually picked, but we see no results. 
Below is the

"rawquerystring":"nasonex",
"querystring":"nasonex",
"parsedquery":"(text:nasonex text:内舒拿)/no_coord",
"parsedquery_toString":"text:nasonex text:内舒拿",
"QParser":"LuceneQParser"


Below is the output when we try to use the analysis tool.

ST

text

raw_bytes

start

end

positionLength

type

position



nasonex

[6e 61 73 6f 6e 65 78]

0

7

1



1



SF

text

raw_bytes

start

end

positionLength

type

position



nasonex

[6e 61 73 6f 6e 65 78]

0

7

1



1


内舒拿

[e5 86 85 e8 88 92 e6 8b bf]

0

7

1

SYNONYM

1



CJKWF

text

raw_bytes

start

end

positionLength

type

position



nasonex

[6e 61 73 6f 6e 65 78]

0

7

1



1


内舒拿

[e5 86 85 e8 88 92 e6 8b bf]

0

7

1

SYNONYM

1



LCF

text

raw_bytes

start

end

positionLength

type

position



nasonex

[6e 61 73 6f 6e 65 78]

0

7

1



1


内舒拿

[e5 86 85 e8 88 92 e6 8b bf]

0

7

1

SYNONYM

1



CJKBF

text

raw_bytes

start

end

positionLength

type

position



nasonex

[6e 61 73 6f 6e 65 78]

0

7

1



1


内舒拿

[e5 86 85 e8 88 92 e6 8b bf]

0

7

1

SYNONYM

1





Please help us regarding this issue. Please let us know if this is the proper 
channel to raise this issue.



Thanks and regards

Santosh Kumar Turangi
MERCK | >Accenture |
Mobile: 08008633009
Email: 
santoshkumar.tura...@accenture.com




This message is for the designated recipient only and may contain privileged, 
proprietary, or otherwise confidential information. If you have received it in 
error, please notify the sender immediately and delete the original. Any other 
use of the e-mail by you is prohibited. Where allowed by local law, electronic 
communications with Accenture and its affiliates, including e-mail and instant 
messaging (including content), may be scanned by our systems for the purposes 
of information security and assessment of internal compliance with Accenture 
policy.
__

www.accenture.com

RE: How to best serialize/deserialize a SolrInputDocument?

[Markus Jelsma]

Hello - we use GZipped output streams too for buffering large sets of 
SolrInputDocument's to disk before indexing. Works fine and SolrInputDocument 
is very easily compressed as well.

Markus

 
 
-Original message-
> From:Sebastian Riemer 
> Sent: Thursday 30th June 2016 13:56
> To: solr-user@lucene.apache.org
> Subject: How to best serialize/deserialize a SolrInputDocument?
> 
> Hi,
> 
> I am looking for a way to serialize a SolrInputDocument.
> 
> I want to store the serialized document in a MySQL table.
> 
> Later I want to deserialize that document and send it to the Solr server.
> 
> Currently I am looking at org.apache.solr.client.solrj.request.UpdateRequest 
> and JavaBinUpdateRequestCodec. There are two methods, marshal and unmarshal 
> which look like I could use for that purpose.
> 
> I'd simply create an UpdateRequest, add the document to it, call marshal, 
> save the OutputStream somehow in the MySQL table. When retrieving I pass the 
> value from the MySQL as InputStream to the unmarshal method, get my 
> UpdateRequest object, iterate the contained SolrInputDocument and send it to 
> the server.
> 
> Am I on the right track, or is there a better approach?
> 
> The background to this is, that we want backup the generated documents which 
> are indexed with solr. So if a client restores a backup, that MySQL table 
> with the serialized documents can be used to rebuild the index as quickly as 
> possible.
> 
> Thanks,
> Sebastian
> 
> 
> 

Re: Solr node crashes while indexing - Too many open files

[Toke Eskildsen]

Mads Tomasgård Bjørgan  wrote:

> That's true, but I was hoping there would be another way to solve this issue 
> as it's not considered preferable in our situation.

What you are looking for might be
https://cwiki.apache.org/confluence/display/solr/IndexConfig+in+SolrConfig#IndexConfiginSolrConfig-CompoundFileSegments

> Is it normal behavior for Solr to open over 4000 files without closing them 
> properly?

Open, yes. Not closing them properly, no. The number of open file handles 
should match the number of files in the index folder.

- Toke Eskildsen, State and University Library, Denmark

Re: puzzling StemmerOverrideFilterFactory

[Dmitry Kan]

Hi,

It appears, the issue was due to a mis-config I did in schema. After
StemmerOverrideFilterFactory was added on both query and index sides, the
problem has disappeared.

Thanks,

Dmitry

On Thu, May 19, 2016 at 9:01 PM, Shawn Heisey  wrote:

> On 5/19/2016 5:26 AM, Dmitry Kan wrote:
> > On query side, right above SOF there is SynonymFilter (SF is not present
> on
> > indexing). It does the following:
> > organization -> organization, organisation
> >
> > SOF turns this pair into: organiz, organis.
>
> Can you put the field and fieldType definitions, plus all files
> referenced in those definitions (like the stemdict.txt file), someplace
> on the Internet we can reach, and give us URL(s) to reach it?  You could
> use gist, http://apaste.info, or similar.  Email attachments often don't
> work on the mailing list, so I don't recommend using them.
>
> If you put an expiration date on whatever you use, make it at least one
> month out.
>
> I see that you mentioned this on IRC as well, EARLY in the morning for
> me.  I will be sporadically checking there.
>
> Thanks,
> Shawn
>
>


-- 
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info

Load a java class on start up

[Mark Robinson]

Hi,

I have a java OBJECT which I need to load once.
I have written a java custom component, which I have added in
"last-components" in solrconfig.xml, from which I want to access the above
mentioned OBJECT when each search request comes in.

Is there a way I can load a java object on server/ instance startup?
OR
Load it when the first call comes to SOLR?

For the time being I created that Java object inside the custom component
itself; but it is loaded each time a search request comes in.

Could some one pls give some pointers on how my above requirement can be
achieved in SOLR?

Thanks!
Mark

Re: Load a java class on start up

[Andrea Gazzarini]

Hi,
the lifecycle of your Solr extension (i.e. the component) is not 
something that's up to you.
Before designing the component you should read the framework docs [1], 
in order to understand the context where it will live, once deployed.


There's nothing, as far as I know, other than the component callbacks 
(e.g. the inform, init methods) that can help you to manage the 
lifecycle of a custom class you're using within the component. Look at 
the SolrCoreAware [2] interface, maybe it could fit your needs.
From what you write it seems you could need something like a singleton 
(which is often an anti-pattern in distributed environment) , but 
without further details I'm just shooting in the dark


In addition: you wrote a component so I guess it shouldn't be so hard to 
have a look at one of the existing built-in components. I'm quite sure 
they already met (and solved) a similar issue.


Best,
Andrea

[1] 
https://lucene.apache.org/solr/6_1_0/solr-core/org/apache/solr/handler/component/SearchComponent.html

[2] https://wiki.apache.org/solr/SolrPlugins#SolrCoreAware

On 30/06/16 16:00, Mark Robinson wrote:

Hi,

I have a java OBJECT which I need to load once.
I have written a java custom component, which I have added in
"last-components" in solrconfig.xml, from which I want to access the above
mentioned OBJECT when each search request comes in.

Is there a way I can load a java object on server/ instance startup?
OR
Load it when the first call comes to SOLR?

For the time being I created that Java object inside the custom component
itself; but it is loaded each time a search request comes in.

Could some one pls give some pointers on how my above requirement can be
achieved in SOLR?

Thanks!
Mark

Re: Remove from list

[Erick Erickson]

see: http://lucene.apache.org/solr/resources.html, there's an
'unsubscribe' link that will automatically do this. NOTE: you _must_
use the exact same e-mail you first subscribed with, this sometimes
trips people up if the mail is forwarded from the original account.

Best,
Erick

On Thu, Jun 30, 2016 at 4:51 AM, Colin Hunter  wrote:
> Hi,
>
> Could you please remove my email from the mailing list (for now).
>
> Many thanks for the help and resource you have provided.
> Colin Hunter
>
> --
> www.gfc.uk.net

Re: Solr Cloud 2nd Server Recover Stuck

[Erick Erickson]

NP, glad it worked!

On Wed, Jun 29, 2016 at 10:33 PM, Tim Chen  wrote:
> Hi Erick,
>
> I have followed your instruction to added as new replica and deleted the old 
> replica - works great!
>
> Everything back to normal now.
>
> Thanks mate!
>
> Cheers,
> Tim
>
> -Original Message-
> From: Erick Erickson [mailto:erickerick...@gmail.com]
> Sent: Thursday, 30 June 2016 1:49 AM
> To: solr-user
> Subject: Re: Solr Cloud 2nd Server Recover Stuck
>
> I'm assuming that 10.1.11.79 is server A here.
>
> What this _looks_ like is that you deleted the entire
> directory here:
> cr_dev_shard1_replica2
> cr_dev_shard2_replica2
>
> but not
> collection1
>
> on server B. This is a little inconsistent, but I think the collection1
> core naming was a little weird with the default collection in 4.10...
>
> Anyway, if this is true then there'll be no
> core.properties
> file in cr_dev_blah blah.
>
> So, Zookeeper has  a record of there being
> such a thing, but it's not present on your sever B.
> To Zookeeper, since the replica hasn't registered
> itself it still looks like the machine is just down.
>
> So here's what I'd try:
> Well, first I'd back up server As index directories...
>
> Use the Collections API ADDREPLICA command to
> add a replica on Server B for each shard, use the "node"
> parameter.
>
> That should churn for a while but eventually create a replica
> and sync it with the leader. Once that's done, use the DELETEREPLICA
> to force Zookeeper to remove the traces of the original replicas on
> server B.
>
> Best,
> Erick
>
> On Wed, Jun 29, 2016 at 12:05 AM, Tim Chen  wrote:
>> Hi,
>>
>> I need some help please.
>>
>> I am running Solr Cloud 4.10.4, with ensemble ZooKeeper.
>>
>> Server A running Solr Cloud + ZooKeeper
>> Server B running Solr Cloud + ZooKeeper
>> Server C running ZooKeeper only.
>>
>> For some reason Server B is crashed and all data lost. I have cleaned it up, 
>> deleted all existing collection index files and start up the Solr service 
>> fresh.
>>
>> If a Collection that has only 1 shard, Server B has managed to create and 
>> replicate from Server A:
>>SolrCore [collection1] Solr index directory 
>> '/collection1/data/index' doesn't exist. Creating new index...
>>
>> If a Collection that has 2 shards, Server B doesn't seem to be doing 
>> anything. The Collection was configured 2 shards and 2 replication 
>> originally.
>>
>> Here is the Clusterstate.json from ZooKeeper.
>>
>> Collection1 has only 1 shard.
>> Collection cr_dev has 2 shards, one is on server A, one was on server B.
>> Server A: 10.1.11.70
>> Server B: 10.2.11.244
>>
>> Is it because "autoCreated" is missing from collection cr_dev? How do I set 
>> this? API call?
>>
>> "collection1":{
>> "shards":{"shard1":{
>> "range":"8000-7fff",
>> "state":"active",
>> "replicas":{
>>   "core_node1":{
>> "state":"active",
>> "core":"collection1",
>> "node_name":"10.1.11.70:8983_solr",
>> "base_url":"http://10.1.11.70:8983/solr";,
>> "leader":"true"},
>>   "core_node2":{
>> "state":"active",
>> "core":"collection1",
>> "node_name":"10.2.11.244:8983_solr",
>> "base_url":"http://10.2.11.244:8983/solr",
>> "maxShardsPerNode":"1",
>> "router":{"name":"compositeId"},
>> "replicationFactor":"1",
>> "autoAddReplicas":"false",
>> "autoCreated":"true"},
>>   "cr_dev":{
>> "shards":{
>>   "shard1":{
>> "range":"8000-",
>> "state":"active",
>> "replicas":{
>>   "core_node1":{
>> "state":"active",
>> "core":"cr_dev_shard1_replica1",
>> "node_name":"10.1.11.70:8983_solr",
>> "base_url":"http://10.1.11.70:8983/solr";,
>> "leader":"true"},
>>   "core_node4":{
>> "state":"down",
>> "core":"cr_dev_shard1_replica2",
>> "node_name":"10.2.11.244:8983_solr",
>> "base_url":"http://10.2.11.244:8983/solr"}}},
>>   "shard2":{
>> "range":"0-7fff",
>> "state":"active",
>> "replicas":{
>>   "core_node2":{
>> "state":"active",
>> "core":"cr_dev_shard2_replica1",
>> "node_name":"10.1.11.70:8983_solr",
>> "base_url":"http://10.1.11.70:8983/solr";,
>> "leader":"true"},
>>   "core_node3":{
>> "state":"down",
>> "core":"cr_dev_shard2_replica2",
>> "node_name":"10.2.11.244:8983_solr",
>> "base_url":"http://10.2.11.244:8983/solr",
>> "maxShardsPerNode":"2",
>> "router":{"name":"compositeId"},
>> "replicationFactor":"2",
>> "autoAddReplicas":"false"},
>>
>> Many thanks,
>> Tim
>>
>>
>> [tour de france 2 july 8:30pm]
>
>
> [tour de france 2 july 8:30pm]

Re: Specify sorting of merged streams

[tedsolr]

I've read about the sort stream in v6.1 but it appears to me to break the
streaming design. If it has to read all the results into memory then it's
not streaming. Sounds like it could be slow and memory intensive for very
large result sets. Has anyone had good results with the sort stream when
there are 10M+ docs returned?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4285202.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Specify sorting of merged streams

[Joel Bernstein]

Hi,

The streaming API in Solr 6x has been expanded to supported many different
parallel computing workloads. For example the topic stream supports pub/sub
messaging. The gatherNodes stream supports graph traversal. The facet
stream supports aggregations inside the search engine, while the rollup
stream supports shuffling map / reduce aggregations. Stored queries and
large scale alerting is on the way...

The sort stream is designed to be used at scale in parallel mode. It can
currently sort about 1,000,000 docs per second on a single worker. So if
you have 20 workers it can sort 20,000,000 docs per second. The plan is to
eventually switch to the fork/join merge sort so that you get parallelism
within the same worker.



Joel Bernstein
http://joelsolr.blogspot.com/

On Thu, Jun 30, 2016 at 3:43 PM, tedsolr  wrote:

> I've read about the sort stream in v6.1 but it appears to me to break the
> streaming design. If it has to read all the results into memory then it's
> not streaming. Sounds like it could be slow and memory intensive for very
> large result sets. Has anyone had good results with the sort stream when
> there are 10M+ docs returned?
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Specify-sorting-of-merged-streams-tp4285026p4285202.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

500 error querying solrcloud with >1 shard

[nuhaa]

hi all,

We have a solrcloud setup with zookeeper, and right now we're testing it
with indexing and querying.


*collection A:*

*collection B:*

​

I'm trying to figure out why on collection B, indexing works but querying
doesn't. I believe by looking at Collection B > Schema > Load Term Info on
a selected field, if I can see the value histogram, that means they're
indexed successfully.

On collection A, indexing and querying both work.

This is what returned when querying on collection B:

{
  "responseHeader":{
"zkConnected":true,
"status":500,
"QTime":17,
"params":{
  "q":"*:*",
  "indent":"on",
  "wt":"json",
  "_":"1467275265841"}},
  "error":{
"trace":"java.lang.NullPointerException\n\tat
org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1209)\n\tat
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:1060)\n\tat
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:759)\n\tat
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:738)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:518)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\n\tat
java.lang.Thread.run(Thread.java:745)\n",
"code":500}}

Is there anything I need to do add on solrconfig.xml?? Or is there
something possibly wrong with the zookeeper/solrcloud configurations?
How can I further investigate this?

Any ideas is much appreciated.


regards,
nuhaa
http://about.me/nuhaa

ImplicitSnitch preferredNodes

[Susheel Kumar]

Hello Arcadius, Noble,

I have a single Solr cluster setup across two DC's having good connectivity
with below similar configuration and looking to use preferredNodes
feature/rule that search queries executed from DC1 client, uses all dc1
replica's and DC2 client, uses all dc2 replica's.

Bit confused with the current documentation (
https://issues.apache.org/jira/browse/SOLR-8522) on what different steps
needs to be taken care on client and zookeeper side?

Can you please summarize what needs to be executed part of Solrj client
configuration/properties and Zookeeper clusterstate (MODIFYCOLLECTION) to
make this work.  In the meantime I'll take a look on tests closely.

DC1 - 3-dc1 shards replica and 3-dc2 shards replica
DC2 - 3-dc2 shards replica and 3-dc1 shards replica


Thanks,
Susheel

Re: Error when searching with special characters

[Zheng Lin Edwin Yeo]

Hi,

When I use defType=edismax, and using debug mode by setting debug=True, I
found that the search for "r&d" is actually done to search on just the
character "r".

http://localhost:8983/solr/collection1/highlight?q=
"r&d"&debugQuery=true&defType=edismax

  "debug":{
"rawquerystring":"\"r",
"querystring":"\"r",
"parsedquery":"(+DisjunctionMaxQuery((text:r)))/no_coord",
"parsedquery_toString":"+(text:r)"


Even if I search with escape character, it is of no help.

http://localhost:8983/solr/collection1/highlight?q=
"r\&d"&debugQuery=true&defType=edismax

  "debug":{
"rawquerystring":"\"r\\",
"querystring":"\"r\\",
"parsedquery":"(+DisjunctionMaxQuery((text:r)))/no_coord",
"parsedquery_toString":"+(text:r)",



But if I'm using other symbols like "r*d", then the search is ok.

http://localhost:8983/solr/collection1/highlight?q=
"r*d"&debugQuery=true&defType=edismax

  "debug":{
"rawquerystring":"\"r*d\"",
"querystring":"\"r*d\"",
"parsedquery":"(+DisjunctionMaxQuery((text:\"r d\")))/no_coord",
"parsedquery_toString":"+(text:\"r d\")",


What could be the reason behind this?


Regards,
Edwin


On 20 June 2016 at 02:12, Ahmet Arslan  wrote:

> Hi,
>
> It is better to create a failing junit test case before opening jira.
>
> ahmet
>
>
> On Sunday, June 19, 2016 4:44 PM, Zheng Lin Edwin Yeo <
> edwinye...@gmail.com> wrote:
>
>
> Yes, it throws the parse exception even if the query is properly escaped
> for ampersand (&) for defType=lucene.
>
> Should we treat this as a bug, and create a JIRA>
>
> Regards,
> Edwin
>
>
>
> On 19 June 2016 at 08:07, Ahmet Arslan  wrote:
>
>
>
> If properly escaped ampersand throws parse exception, this could be a bug.
>
>
>
> On Saturday, June 18, 2016 7:12 PM, Zheng Lin Edwin Yeo <
> edwinye...@gmail.com> wrote:
> Hi,
>
> It does not work with the back slash too.
>
> But I found that it does not work for defType=lucene.
> It will work if the defType=dismax or edismax.
>
> What could be the reason that it did not work with the default
> defType=lucene?
>
> Regards,
> Edwin
>
>
>
> On 18 June 2016 at 01:04, Ahmet Arslan  wrote:
>
> > Hi,
> >
> > May be URL encoding issue?
> > By the way, I would use back slash to escape special characters.
> >
> > Ahmet
> >
> > On Friday, June 17, 2016 10:08 AM, Zheng Lin Edwin Yeo <
> > edwinye...@gmail.com> wrote:
> >
> >
> >
> > Hi,
> >
> > I encountered this error when I tried to search with special characters,
> > like "&" and "#".
> >
> > {
> >   "responseHeader":{
> > "status":400,
> > "QTime":0},
> >   "error":{
> > "msg":"org.apache.solr.search.SyntaxError: Cannot parse
> > '\"Research ': Lexical error at line 1, column 11.  Encountered: 
> > after : \"\\\"Research \"",
> > "code":400}}
> >
> >
> > I have done the search by putting inverted commands, like: q="Research &
> > Development"
> >
> > What could be the issue here?
> >
> > I'm facing this problem in both Solr 5.4.0 and Solr 6.0.1.
> >
> >
> > Regards,
> > Edwin
> >
>
>
>
>
>

Re: 500 error querying solrcloud with >1 shard

[Erick Erickson]

None of the pasted images came through, the mail server is quite aggressive
about stripping them. You'll need to upload them somewhere and provide a
link.

Best,
Erick

On Thu, Jun 30, 2016 at 7:36 PM, nuhaa  wrote:

> hi all,
>
> We have a solrcloud setup with zookeeper, and right now we're testing it
> with indexing and querying.
>
>
> *collection A:*
>
> *collection B:*
>
> ​
>
> I'm trying to figure out why on collection B, indexing works but querying
> doesn't. I believe by looking at Collection B > Schema > Load Term Info on
> a selected field, if I can see the value histogram, that means they're
> indexed successfully.
>
> On collection A, indexing and querying both work.
>
> This is what returned when querying on collection B:
>
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":500,
> "QTime":17,
> "params":{
>   "q":"*:*",
>   "indent":"on",
>   "wt":"json",
>   "_":"1467275265841"}},
>   "error":{
> "trace":"java.lang.NullPointerException\n\tat
> org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1209)\n\tat
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:1060)\n\tat
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:759)\n\tat
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:738)\n\tat
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)\n\tat
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\n\tat
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\n\tat
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\n\tat
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:518)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\n\tat
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\n\tat
> java.lang.Thread.run(Thread.java:745)\n",
> "code":500}}
>
> Is there anything I need to do add on solrconfig.xml?? Or is there
> something possibly wrong with the zookeeper/solrcloud configurations?
> How can I further investigate this?
>
> Any ideas is much appreciated.
>
>
> regards,
> nuhaa
> http://about.me/nuhaa
>

Re: 500 error querying solrcloud with >1 shard

[Nuhaa All Bakry]

ah i see

https://ibin.co/2mWaie7IVxDF.png

Collection A:
https://ibin.co/2mWawhCm76cN.png

Collection B:
https://ibin.co/2mWb4BmlMom2.png



--
nuhaa
http://about.me/nuhaa

> On 1 Jul 2016, at 12:24 PM, Erick Erickson  wrote:
> 
> None of the pasted images came through, the mail server is quite aggressive
> about stripping them. You'll need to upload them somewhere and provide a
> link.
> 
> Best,
> Erick
> 
> On Thu, Jun 30, 2016 at 7:36 PM, nuhaa  wrote:
> 
>> hi all,
>> 
>> We have a solrcloud setup with zookeeper, and right now we're testing it
>> with indexing and querying.
>> 
>> 
>> *collection A:*
>> 
>> *collection B:*
>> 
>> ​
>> 
>> I'm trying to figure out why on collection B, indexing works but querying
>> doesn't. I believe by looking at Collection B > Schema > Load Term Info on
>> a selected field, if I can see the value histogram, that means they're
>> indexed successfully.
>> 
>> On collection A, indexing and querying both work.
>> 
>> This is what returned when querying on collection B:
>> 
>> {
>>  "responseHeader":{
>>"zkConnected":true,
>>"status":500,
>>"QTime":17,
>>"params":{
>>  "q":"*:*",
>>  "indent":"on",
>>  "wt":"json",
>>  "_":"1467275265841"}},
>>  "error":{
>>"trace":"java.lang.NullPointerException\n\tat
>> org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1209)\n\tat
>> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:1060)\n\tat
>> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:759)\n\tat
>> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:738)\n\tat
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)\n\tat
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\n\tat
>> org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\n\tat
>> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\n\tat
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\n\tat
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\n\tat
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\n\tat
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\n\tat
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\n\tat
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\n\tat
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
>> org.eclipse.jetty.server.Server.handle(Server.java:518)\n\tat
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\n\tat
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\n\tat
>> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat
>> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat
>> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\n\tat
>> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\n\tat
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\n\tat
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\n\tat
>> java.lang.Thread.run(Thread.java:745)\n",
>>"code":500}}
>> 
>> Is there anything I need to do add on solrconfig.xml?? Or is there
>> something possibly wrong with the zookeeper/solrcloud configurations?
>> How can I further investigate this?
>> 
>> Any ideas is much appreciated.
>> 
>> 
>> regards,
>> nuhaa
>> http://about.me/nuhaa
>>