BigDecimal Solr Field in schema
Hello All, I am using Solr 6.0.0 in cloud mode and have requirement to support all number in BigDecimal Does anyone know which solr field type should be used for BigDecimal? I tried using DoubleTrieField but it does not meet the requirement and round up very big number approx. after 16 digit. Regards, Roshan The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
CloudSolrStream returns only top 30 results.
Hello, I have a plan to use streaming API for join queries. But it has been observed that CloudSolrStream returns only top 30 matching records. Is there any configuration which needs to be done to retrieve ALL records or to have pagination like provision to specify start or rows attributes? Regards, Roshan The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
SolrCloud 6 Join Stream and pagination
Hello, I am using Solr 6 in cloud mode. In order to search within different collections I am using InnerJoinStream. (using qt=export in order to get correct result) Is there any way to get paginated result? Regards, Roshan The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
wildcard search for string having spaces
Hello, I have below custom field type defined for solr 6.0.0 I am using above field to ensure that entire string is considered as single token and search should be case insensitive. It works for most of the scnearios with wildcard search. e.g. if my data is "abc.pqr" and "abc_pqr" and "abc pqr" then search with abc* gives this three results. But I am not able to search with say abc p* Search with query q="abc pqr" gives exact match and desired result. I want to do wildcard search where criteria can include spaces like above example i.e. if space is present then I am not able to to wildcard search. Is there any way by which wildcard search will be achieved even if space is present in token. Regards, Roshan The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
RE: wildcard search for string having spaces
Great.
First option worked for me. I was trying with q=abc\sp*... it should be q=abc\
p*
Thanks
-Original Message-
From: Ahmet Arslan [mailto:iori...@yahoo.com]
Sent: Wednesday, June 15, 2016 6:25 PM
To: solr-user@lucene.apache.org; Roshan Kamble
Subject: Re: wildcard search for string having spaces
Hi Roshan,
I think there are two options:
1) escape the space q=abc\ p*
2) use prefix query parser q={!prefix f=my_string}abc p
Ahmet
On Wednesday, June 15, 2016 3:48 PM, Roshan Kamble
wrote:
Hello,
I have below custom field type defined for solr 6.0.0
I am using above field to ensure that entire string is considered as single
token and search should be case insensitive.
It works for most of the scnearios with wildcard search.
e.g. if my data is "abc.pqr" and "abc_pqr" and "abc pqr" then search with abc*
gives this three results.
But I am not able to search with say abc p*
Search with query q="abc pqr" gives exact match and desired result.
I want to do wildcard search where criteria can include spaces like above
example
i.e. if space is present then I am not able to to wildcard search.
Is there any way by which wildcard search will be achieved even if space is
present in token.
Regards,
Roshan
The information in this email is confidential and may be legally privileged. It
is intended solely for the addressee. Access to this email by anyone else is
unauthorised. If you are not the intended recipient, any disclosure, copying,
distribution or any action taken or omitted to be taken in reliance on it, is
prohibited and may be unlawful.
The information in this email is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this email by anyone else is
unauthorised. If you are not the intended recipient, any disclosure, copying,
distribution or any action taken or omitted to be taken in reliance on it, is
prohibited and may be unlawful.
SolrCloud persisting data is very slow
Hello, I am using Solr 6.0.0 in cloudMode (3 physical nodes + one zookeeper) and have heavy insert/update/delete operations. I am using CloudSolrClient and tried with all batch size from 100 to 1000. But it has been observed that persist at Solr node is very slow. It takes around 20 secords to store 50-100 records. Does anyone know how to improve the speed for these operations? Regards, Roshan The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
Could not load collection for SolrCloud
Hello, I am using solr 6.0.0 in SolCloud mode with 3 nodes, one zookeeper and 3 shard and 2 replica per collection. Getting below error for some insert/update when trying to insert documents to Solr. And it has been observed that few shard are in either recovery or fail recovery state. (Atleast one shard is up) org.apache.solr.common.SolrException: Could not load collection from ZK: MY_COLLECTION at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:969) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:519) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:189) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.ClusterState.hasCollection(ClusterState.java:119) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:833) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_60] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_60] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_60] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60] Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/ MY_COLLECTION /state.json at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) ~[zookeeper-3.4.6.jar:3.4.6-1569965] at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) ~[zookeeper-3.4.6.jar:3.4.6-1569965] at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) ~[zookeeper-3.4.6.jar:3.4.6-1569965] at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:980) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:967) ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - nknize - 2016-04-01 14:41:50] ... 16 more Regards, Roshan The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
RE: Could not load collection for SolrCloud
Hello Erick, i have four collection in solrcloud. there are heavy insert on each collection, but below issue is observed for one collection. I tried increasing zk session timeout upto 2 mins but no luck. Not sure about reason of session timeout at zookeeper when insert is made. do we need to explicitly call any API to avoid session timwout? From: Erick Erickson [erickerick...@gmail.com] Sent: Sunday, June 26, 2016 9:32 PM To: solr-user Subject: Re: Could not load collection for SolrCloud My guess is that you are hitting garbage collection issues on those shards that are going into recovery. If a leader tries to contact a follower in a single shard and times out it effectively says "that one must be gone, let's put it into recovery". Look for LeaderInitiatedRecovery (don't remember whether there are spaces etc. in that though) in the Solr logs on both the leader and follower. Next I'd turn on GC logging and look for stop-the-world recovery events that take a long time, GCViewer is a nice tool for looking at those IIRC. Zookeeper also periodically pings the Solr nodes and if ZK can't get a response (again possibly due to excessive GC) it'll signal that the node is down. If that happens,though, I'd expect multiple replicas on a particular Solr instance to go into recovery. And finally you can consider lengthening the timeouts Best, Erick On Sat, Jun 25, 2016 at 1:18 PM, Roshan Kamble wrote: > Hello, > > I am using solr 6.0.0 in SolCloud mode with 3 nodes, one zookeeper and 3 > shard and 2 replica per collection. > > Getting below error for some insert/update when trying to insert documents to > Solr. > > And it has been observed that few shard are in either recovery or fail > recovery state. (Atleast one shard is up) > > > org.apache.solr.common.SolrException: Could not load collection from ZK: > MY_COLLECTION > at > org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:969) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:519) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:189) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.common.cloud.ClusterState.hasCollection(ClusterState.java:119) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:833) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71) > ~[solr-solrj-6.0.0.jar:6.0.0 48c80f91b8e5cd9b3a9b48e6184bd53e7619e7e3 - > nknize - 2016-04-01 14:41:50] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [?:1.8.0_60] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [?:1.8.0_60] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_60] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_60] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60] > Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: > KeeperErrorCode = Session expired for /collections/ MY_COLLECTION /state.json > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:127) > ~[zookeeper-3.4.6.jar:3.4.6-1569965] >
IO Exception : Truncated chunk for WORKER collection for paraller stream Join Query
Hello, we are using Solr 6.0.0 in CloudMode with 3 physical nodes and 3 shards per collection. we are using ParallelStream for our join searches. Below error is observed when while searching with join query. java.util.concurrent.ExecutionException: java.io.IOException: --> http://XX:XX:XX:XX:8983/solr/WORKER_shard3_replica2/:Truncated chunk ( expected size: 32768; actual size: 11096) at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:355) ... 70 more Caused by: java.io.IOException: --> http://XX:XX:XX:XX?:8983/solr/WORKER_shard3_replica2/:Truncated chunk ( expected size: 32768; actual size: 11096) at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:201) at org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:449) at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:472) at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:459) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ... 1 more Regards, Roshan Kamble The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
RE: IO Exception : Truncated chunk for WORKER collection for paraller stream Join Query
...@gmail.com] Sent: Thursday, June 30, 2016 1:02 AM To: solr-user@lucene.apache.org Subject: Re: IO Exception : Truncated chunk for WORKER collection for paraller stream Join Query Does this log come from the worker node or from the shards? It would be good to see the error logs from the server that this node was talking to when the exception was thrown. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jun 28, 2016 at 1:06 PM, Roshan Kamble < roshan.kam...@smartstreamrdu.com> wrote: > Hello, > > we are using Solr 6.0.0 in CloudMode with 3 physical nodes and 3 > shards per collection. > > > we are using ParallelStream for our join searches. > > > Below error is observed when while searching with join query. > > > java.util.concurrent.ExecutionException: java.io.IOException: --> > http://XX:XX:XX:XX:8983/solr/WORKER_shard3_replica2/:Truncated chunk ( > expected size: 32768; actual size: 11096) at > java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(Clo > udSolrStream.java:355) > ... 70 more > Caused by: java.io.IOException: --> > http://XX:XX:XX:XX?:8983/solr/WORKER_shard3_replica2/:Truncated > chunk ( expected size: 32768; actual size: 11096) at > org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java > :201) > at > org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.ne > xt(CloudSolrStream.java:449) > at > org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.ca > ll(CloudSolrStream.java:472) > at > org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.ca > ll(CloudSolrStream.java:459) at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.la > mbda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j > ava:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > java:617) > ... 1 more > > > Regards, > Roshan Kamble > > > > The information in this email is confidential and may be legally > privileged. It is intended solely for the addressee. Access to this > email by anyone else is unauthorised. If you are not the intended > recipient, any disclosure, copying, distribution or any action taken > or omitted to be taken in reliance on it, is prohibited and may be unlawful. > The information in this email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
