Re: Idle Timeout while DIH indexing and implicit sharding in 7.4

2019-09-03 Thread Mikhail Khludnev
Message- > > From: Mikhail Khludnev [mailto:m...@apache.org] > > Sent: Monday, September 02, 2019 12:23 PM > > To: Vadim Ivanov; solr-user > > Subject: Re: Idle Timeout while DIH indexing and implicit sharding in 7.4 > > > > It seems like reasonable behavior. Solr

RE: Idle Timeout while DIH indexing and implicit sharding in 7.4

2019-09-02 Thread Vadim Ivanov
ccept records from DIH. Am I wrong? -- Vadim > -Original Message- > From: Mikhail Khludnev [mailto:m...@apache.org] > Sent: Monday, September 02, 2019 12:23 PM > To: Vadim Ivanov; solr-user > Subject: Re: Idle Timeout while DIH indexing and implicit sharding in 7.4 > >

Re: Idle Timeout while DIH indexing and implicit sharding in 7.4

2019-09-02 Thread Mikhail Khludnev
@apache.org] > *Sent:* Monday, September 02, 2019 1:31 AM > *To:* solr-user > *Cc:* vadim.iva...@spb.ntk-intourist.ru > *Subject:* Re: Idle Timeout while DIH indexing and implicit sharding in > 7.4 > > > > Giving that > > org.apache.solr.common.util.FastInputStream.peek(

Re: Idle Timeout while DIH indexing and implicit sharding in 7.4

2019-09-01 Thread Mikhail Khludnev
Giving that org.apache.solr.common.util.FastInputStream.peek(FastInputStream.java:60) at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDoc s(JavabinLoader.java:107) JavabinLoader hangs on Stream.peek(), awaiting -1, and hit timeout. I guess it's might be related with "closing so

RE: Idle Timeout while DIH indexing and implicit sharding in 7.4

2019-09-01 Thread swapna.minnaka...@copart.com
I am facing same exact issue. We never had any issue with 6.5.1 when doing full index (initial bulk load) After upgrading to 7.5.0, getting below exception and indexing is taking a very long time 2019-09-01 10:11:27.436 ERROR (qtp1650813924-22) [c:c_collection s:shard1 r:core_node3 x:c_collection_

RE: Idle Timeout while DIH indexing and implicit sharding in 7.4

2019-09-01 Thread swapna.minnaka...@copart.com
I am facing same exact issue. We never had any issue with 6.5.1 when doing full index (initial bulk load) After upgrading to 7.5.0, getting below exception and indexing is taking a very long time 2019-09-01 10:11:27.436 ERROR (qtp1650813924-22) [c:c_member_lots_a s:shard1 r:core_node3 x:c_collecti

RE: Idle Timeout while DIH indexing and implicit sharding in 7.4

2018-09-14 Thread Vadim Ivanov
t: Friday, September 14, 2018 12:10 PM To: solr-user Subject: Re: Idle Timeout while DIH indexing and implicit sharding in 7.4 Hello, Vadim. My guess (and only guess) that bunch of updates coming into a shard causes a heavy merge that blocks new updates in its' order. This can be veri

Re: Idle Timeout while DIH indexing and implicit sharding in 7.4

2018-09-14 Thread Mikhail Khludnev
Hello, Vadim. My guess (and only guess) that bunch of updates coming into a shard causes a heavy merge that blocks new updates in its' order. This can be verified with logs or threaddump from the problematic node. The probable measures are: try to shuffle updates to load other shards for a while an

RE: Idle Timeout while DIH indexing and implicit sharding in 7.4

2018-09-13 Thread Vadim Ivanov
Hi, I've put some more tests on the issue and managed to find out more details. Time out occurs when while long indexing some documents in the beginning is going to one shard and then for a long time (more than 120 sec) no data at all is going to that shard. Connection to that core, opened in the b