Can you please provide the whole update chain ? Below is the directoryFactory
and not the update processor chain.
> Am 15.11.2019 um 17:34 schrieb Fiz N <fiznewy...@gmail.com>:
>
>
> Thanks for your response.
>
> <directoryFactory
> name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
> attaching the screenshot of physical memory and cpu.
> Please let me know your thoughts on the below issue.
>
>
>> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <jornfra...@gmail.com> wrote:
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An
>> application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <fiznewy...@gmail.com>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch size
>> > per thread 50 & 100, it worked fine without error and if the batch size is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
>> > respond
>> >
>> > at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> > at
>> > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> > at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> > at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> > at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > -
>> > null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> > at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> > at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
