Some more information on this... Most of documents get indexed properly. Few
documents are not getting indexed.
All documents POST are seen in the localhost_access and 200 OK response is seen
in local host access file. But in catalina, there are some difference in the
logs for which are indexing properly, following is the logs.
FINE: PRE_UPDATE add
{,id=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001}
params(crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001),defaults(wt=xml)
Sep 01, 2016 7:39:31 AM org.apache.solr.update.TransactionLog <init>
FINE: New TransactionLog
file=/ebdata2/solrdata/IOB_shard1_replica1/data/tlog/tlog.0000000000000220856,
exists=false, size=0, openExisting=false
Sep 01, 2016 7:39:31 AM org.apache.solr.update.SolrCmdDistributor submit
FINE: sending update to http://xx.xx.xx.xx:7070/solr/IOB_shard1_replica2/
retry:0
add{version=1544254202941800448,id=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001}
params:update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fxx.xx.xx.xx%3A7070%2Fsolr%2FIOB_shard1_replica1%2F
Sep 01, 2016 7:39:31 AM
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner run
FINE: starting runner:
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner@3fb794b2
Sep 01, 2016 7:39:31 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
FINE: PRE_UPDATE FINISH
params(crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001),defaults(wt=xml)
Sep 01, 2016 7:39:31 AM
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner run
FINE: finished:
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner@3fb794b2
Sep 01, 2016 7:39:31 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [IOB_shard1_replica1] webapp=/solr path=/update params=
{crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001}
{add=[CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001
(1544254202941800448)]}
Sep 01, 2016 7:39:31 AM org.apache.solr.servlet.SolrDispatchFilter doFilter
FINE: Closing out SolrRequest:
params(crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001),defaults(wt=xml)
-------------------------------------------------
For the one which document is not getting indexed, we could see only following
log in catalina.out. Not sure whether it's getting added to SOLR.
Sep 01, 2016 7:39:56 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
FINE: PRE_UPDATE FINISH
params(crid=CUA0000004390000019223370564139182810C3LEA0000020179223370567061972057EXCC1000002),defaults(wt=xml)
Sep 01, 2016 7:39:56 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [IOB_shard1_replica1] webapp=/solr path=/update params=
{crid=CUA0000004390000019223370564139182810C3LEA0000020179223370567061972057EXCC1000002}
{} 0 1
Sep 01, 2016 7:39:56 AM org.apache.solr.servlet.SolrDispatchFilter doFilter
FINE: Closing out SolrRequest:
params(crid=CUA0000004390000019223370564139182810C3LEA0000020179223370567061972057EXCC1000002),defaults(wt=xml)
----------------------
You can see that in above log for missing documents ( which is not indexed), in
catalina log, we are not seeing "PRE UPDATE ADD". Is that causing / reason for
document not getting indexed ?
We have set autosoftcommit to 1 seconds and autohardcommit to 30 seconds.
We are not getting any errors or exceptions in the log.
This issue is becoming very critical and sort of reliable factor. Though we get
200 OK response from SOLR for update HTTP POST request, nothing happens on the
SOLR side. If SOLR is not able to process, isn't it we get error from SOLR
instead of giving 200 OK response.
Anybody has faced this sort of issue or any sort of help would be very much
appreciated.
On Sun, Sep 4, 2016 at 12:59 PM Ganesh M
<mgane...@live.in<mailto:mgane...@live.in>> wrote:
Nitin, Thanks for reply. Our each document has unique id and its hbase rowkey
id. So it will be unique only. So there is no chance of duplicates id being
send.
On Sun 4 Sep, 2016 12:41 pm Nitin Kumar,
<nitinkumar.i...@gmail.com<mailto:nitinkumar.i...@gmail.com>> wrote:
Please check doc's unique key(Id). All keys shd be unique. Else docs having
same id will be replaced.
On 04-Sep-2016 12:13 PM, "Ganesh M" <mgane...@live.in<mailto:mgane...@live.in>>
wrote:
> Hi,
> we are keep sending documents to Solr from our app server. Single document
> per request, but in parallel of 10 request hits solr cloud in a second.
>
> We could see our post request ( update request ) hitting our solr 5.4 in
> localhost_access logs, and it's response as 200 Ok response. And also we
> get HTTP 200 OK response to our app servers as well for out HTTP request we
> fired to SOLR Cloud.
>
> But few documents are not getting indexed. Out of 2000 documents we sent
> 10 documents are getting missed. Thought there is not error, few documents
> are getting missed.
>
> We use autoSoftcommit as 2 secs and autohardcommit as 30 secs.
>
> Why is that 10 documents not getting indexed and also no error getting
> thrown back if server is not able to index it ?
>
> Regards,
>
>
>
>
