Re: Regarding LTR feature

2018-05-07 Thread prateek . agarwal
Hi Alessandro, You're right it doesn't have to be that accurate to the query time but our requirement is having a more solid control over our outputs from Solr like if we have 4 features then we can adjust the weights giving something like (40,20,20,20) to each feature such that the sum total o

solr collection id field type long

2018-05-07 Thread Vincenzo D'Amore
Hi all, I'm moving an old collection from Solr 4.8.1 to 7.3.0 where the "id" field has type solr.TrieLongField. Given that solr.TrieLongField has been deprecated I've changed to with the newer LongPointField. But when I tried to create the collection Solr returned the following exception: org.a

Re: Howto disable PrintGCTimeStamps in Solr

2018-05-07 Thread Dominique Bejean
Hi, Which version of Solr are you using ? Regards Dominique Le ven. 4 mai 2018 à 09:13, Bernd Fehling a écrit : > Hi list, > > this sounds simple but I can't disable PrintGCTimeStamps in solr_gc > logging. > I tried with GC_LOG_OPTS in start scripts and --verbose reporting during > start to

Search Help

2018-05-07 Thread natejasper
Hello all here, I'm setting up SOLR on an internal website for my company and I would like to know if anyone can recommend an analytics that I can see what the users are searching for? Does the log in SOLR give me that information? Thanks for your time, -- Sent from: http://lucene.472066.n3.n

Re: solr collection id field type long

2018-05-07 Thread Shawn Heisey
On 5/7/2018 3:27 AM, Vincenzo D'Amore wrote: So just to understand, why we have this behaviour? Is there anything, a mail thread or a ticket I could read? https://issues.apache.org/jira/browse/SOLR-10829?attachmentOrder=desc Thanks, Shawn

Re: solr collection id field type long

2018-05-07 Thread Vincenzo D'Amore
Thanks :) On Mon, May 7, 2018 at 4:18 PM, Shawn Heisey wrote: > On 5/7/2018 3:27 AM, Vincenzo D'Amore wrote: > >> So just to understand, why we have this behaviour? Is there anything, a >> mail thread or a ticket I could read? >> > > https://issues.apache.org/jira/browse/SOLR-10829?attachmentOrd

Re: Search Help

2018-05-07 Thread Shawn Heisey
On 5/7/2018 8:09 AM, natejasper wrote: I'm setting up SOLR on an internal website for my company and I would like to know if anyone can recommend an analytics that I can see what the users are searching for? Does the log in SOLR give me that information? Unless the logging configuration is chan

Re: Howto disable PrintGCTimeStamps in Solr

2018-05-07 Thread Bernd Fehling
Hi Dominique, thanks for asking, I figured it out this morning. If setting -Xloggc= the option -XX:+PrintGCTimeStamps will be set as default and can't be disabled. It's inside JAVA. Currently using Solr 6.4.2 with Java HotSpot(TM) 64-Bit Server VM (25.121-b13) for linux-amd64 JRE (1.8.0_121-b13)

Re: ampersand handling in solr cloud 7 in text_general field

2018-05-07 Thread kumar gaurav
Hi Shawn It is solr 7.3 . On Sun, May 6, 2018 at 1:17 AM, Shawn Heisey wrote: > On 5/5/2018 1:02 PM, kumar gaurav wrote: > >> I am facing possible analysis error. in case of indexing "&" ( ampersand ) >> in text_general fields . It is working fine if solr is running in single >> node mode also

Re: ampersand handling in solr cloud 7 in text_general field

2018-05-07 Thread Shawn Heisey
On 5/7/2018 8:45 AM, kumar gaurav wrote: Hi Shawn It is solr 7.3 . On Sun, May 6, 2018 at 1:17 AM, Shawn Heisey wrote: The error in what you shared is incomplete. Can you find any errors in solr.log and provide the full error text for any of them that occurred around the relevant timestamp?

Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
Hi, I am seeing the following lines in the error log. My setup has 2 nodes in the solrcloud cluster, each node has 3 shards with no replication. From the error log it seems like all the shards on this box are throwing async exception errors. Other node in the cluster does not have any errors in the

Re: Async exceptions during distributed update

2018-05-07 Thread Emir Arnautović
Hi Jay, My first guess would be that there was some major GC on other box so it did not respond on time. Are your nodes well balanced - do they serve equal amount of data? Thanks, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
Yes, the nodes are well balanced. I am just using these boxes for indexing the data and is not serving any traffic at this time. The error indicates it is having issues errors on the shards that are hosted on the box and not on the other box. I will check GC logs to see if there were any issues. t

Re: Async exceptions during distributed update

2018-05-07 Thread Emir Arnautović
Node A receives batch of documents to index. It forwards documents to shards that are on the node B. Node B is having issues with GC so it takes a while to respond. Node A sees it as read timeout and reports it in logs. So the issue is on node B not node A. Emir -- Monitoring - Log Management

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
Ah thanks for explaining that! Thanks Jay Potharaju On Mon, May 7, 2018 at 9:45 AM, Emir Arnautović < emir.arnauto...@sematext.com> wrote: > Node A receives batch of documents to index. It forwards documents to > shards that are on the node B. Node B is having issues with GC so it takes > a whi

Must clause with filter queries

2018-05-07 Thread manuj singh
Hi all, I am kind of confused how must clause(+) behaves with the filter queries. e.g i have below query: q=*:*&fq=+{!frange cost=200 l=NOW-179DAYS u=NOW/DAY+1DAY incl=true incu=false}date So i am filtering documents which are less then 179 old days. So e.g if now is May 7th, 10.23 cst,2018, i sho

Re: Determine Solr Core Creation Timestamp

2018-05-07 Thread Atita Arora
Hi Shawn, I noticed the same and hence overruled the idea to use it. Further , while exploring the V2 api (as we're currently in Solr 6.6 and will soon be on Solr 7.X) ,I came across the shards API which has "property.index.version": "1525453818563" Which is listed for each of the shards. I wonde

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
I didn't see any OOM errors in the logs on either of the nodes. I saw GC pause of 1 second on the box that was throwing error ...but nothing on the other node. Any other recommendations? Thanks Thanks Jay Potharaju On Mon, May 7, 2018 at 9:48 AM, Jay Potharaju wrote: > Ah thanks for explainin

Re: Async exceptions during distributed update

2018-05-07 Thread Emir Arnautović
How do you send documents? Large batches? Complex analysis? Do you send all batches to the same node? How do you commit? Do you delete by query while indexing? Emir On Tue, May 8, 2018, 12:30 AM Jay Potharaju wrote: > I didn't see any OOM errors in the logs on either of the nodes. I saw GC > pa

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
The updates are pushed in real time not batched. No complex analysis and everything is committed using autocommit settings in solr. Thanks Jay Potharaju On Mon, May 7, 2018 at 4:00 PM, Emir Arnautović < emir.arnauto...@sematext.com> wrote: > How do you send documents? Large batches? Complex ana

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
There are some deletes by query. I have not had any issues with DBQ, currently have 5.3 running in production. Thanks Jay Potharaju On Mon, May 7, 2018 at 4:02 PM, Jay Potharaju wrote: > The updates are pushed in real time not batched. No complex analysis and > everything is committed using au

Re: Async exceptions during distributed update

2018-05-07 Thread Emir Arnautović
How many concurrent updates can be sent? Do you always send updates to the same node? Do you use solrj? Emir On Tue, May 8, 2018, 1:02 AM Jay Potharaju wrote: > The updates are pushed in real time not batched. No complex analysis and > everything is committed using autocommit settings in solr.

Re: Async exceptions during distributed update

2018-05-07 Thread Shawn Heisey
On 5/7/2018 5:05 PM, Jay Potharaju wrote: > There are some deletes by query. I have not had any issues with DBQ, > currently have 5.3 running in production. Here's the big problem with DBQ.  Imagine this sequence of events with these timestamps: 13:00:00: A commit for change visibility happens. 1

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
Thanks for explaining that Shawn! Emir, I use php library called solarium to do updates/deletes to solr. The request is sent to any of the available nodes in the cluster. > On May 7, 2018, at 5:02 PM, Shawn Heisey wrote: > >> On 5/7/2018 5:05 PM, Jay Potharaju wrote: >> There are some deletes b

Re: Async exceptions during distributed update

2018-05-07 Thread Jay Potharaju
I have about 3-5 updates per second. > On May 7, 2018, at 5:02 PM, Shawn Heisey wrote: > >> On 5/7/2018 5:05 PM, Jay Potharaju wrote: >> There are some deletes by query. I have not had any issues with DBQ, >> currently have 5.3 running in production. > > Here's the big problem with DBQ. Imagi

LTR performance issues

2018-05-07 Thread ilayaraja
LTR with grouping results in very high latency (3x) even while re-ranking 24 top groups. How is re-ranking implemented in Solr? Is it expected that it would result in 3x more query time. Need clarifications on: 1. How many top groups are actually re-ranked, is it exactly what we pass in reRankDoc

LTR performance issues

2018-05-07 Thread ilayaraja
LTR with grouping results in very high latency (3x) even while re-ranking 24 top groups. How is re-ranking implemented in Solr? Is it expected that it would result in 3x more query time. Need clarifications on: 1. How many top groups are actually re-ranked, is it exactly what we pass in reRankDoc