Can you identify the specific documents that 'fail'? What happens if you post them manually? Try posting them manually but with one field super-distinct to see whether it made it in. What happens if you post it to an empty index (copy definition and try).
Also, what's your request handler's parameters look like. Perhaps you have a signature processor, in which case it may be triggering duplicates avoidance with different calculation from just an id. My guess is still that it is some sort of duplicate issue. Regards, Alex. ---- Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.com/ On 4 September 2016 at 23:10, Ganesh M <mgane...@live.in> wrote: > Some more information on this... Most of documents get indexed properly. Few > documents are not getting indexed. > > All documents POST are seen in the localhost_access and 200 OK response is > seen in local host access file. But in catalina, there are some difference in > the logs for which are indexing properly, following is the logs. > > FINE: PRE_UPDATE add > {,id=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001} > params(crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001),defaults(wt=xml) > Sep 01, 2016 7:39:31 AM org.apache.solr.update.TransactionLog <init> > FINE: New TransactionLog > file=/ebdata2/solrdata/IOB_shard1_replica1/data/tlog/tlog.0000000000000220856, > exists=false, size=0, openExisting=false > Sep 01, 2016 7:39:31 AM org.apache.solr.update.SolrCmdDistributor submit > FINE: sending update to http://xx.xx.xx.xx:7070/solr/IOB_shard1_replica2/ > retry:0 > add{version=1544254202941800448,id=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001} > > params:update.distrib=FROMLEADER&distrib.from=http%3A%2F%2Fxx.xx.xx.xx%3A7070%2Fsolr%2FIOB_shard1_replica1%2F > Sep 01, 2016 7:39:31 AM > org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner run > FINE: starting runner: > org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner@3fb794b2 > Sep 01, 2016 7:39:31 AM org.apache.solr.update.processor.LogUpdateProcessor > finish > FINE: PRE_UPDATE FINISH > params(crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001),defaults(wt=xml) > Sep 01, 2016 7:39:31 AM > org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner run > FINE: finished: > org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner@3fb794b2 > Sep 01, 2016 7:39:31 AM org.apache.solr.update.processor.LogUpdateProcessor > finish > INFO: [IOB_shard1_replica1] webapp=/solr path=/update params= > {crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001} > {add=[CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001 > (1544254202941800448)]} > Sep 01, 2016 7:39:31 AM org.apache.solr.servlet.SolrDispatchFilter doFilter > FINE: Closing out SolrRequest: > params(crid=CUA0000004390000019223370564139207241C3LEA0000020769223370567404392838EXCC3000001),defaults(wt=xml) > ------------------------------------------------- > > For the one which document is not getting indexed, we could see only > following log in catalina.out. Not sure whether it's getting added to SOLR. > > > Sep 01, 2016 7:39:56 AM org.apache.solr.update.processor.LogUpdateProcessor > finish > FINE: PRE_UPDATE FINISH > params(crid=CUA0000004390000019223370564139182810C3LEA0000020179223370567061972057EXCC1000002),defaults(wt=xml) > Sep 01, 2016 7:39:56 AM org.apache.solr.update.processor.LogUpdateProcessor > finish > INFO: [IOB_shard1_replica1] webapp=/solr path=/update params= > {crid=CUA0000004390000019223370564139182810C3LEA0000020179223370567061972057EXCC1000002} > {} 0 1 > Sep 01, 2016 7:39:56 AM org.apache.solr.servlet.SolrDispatchFilter doFilter > FINE: Closing out SolrRequest: > params(crid=CUA0000004390000019223370564139182810C3LEA0000020179223370567061972057EXCC1000002),defaults(wt=xml) > > ---------------------- > > You can see that in above log for missing documents ( which is not indexed), > in catalina log, we are not seeing "PRE UPDATE ADD". Is that causing / reason > for document not getting indexed ? > > We have set autosoftcommit to 1 seconds and autohardcommit to 30 seconds. > > We are not getting any errors or exceptions in the log. > > This issue is becoming very critical and sort of reliable factor. Though we > get 200 OK response from SOLR for update HTTP POST request, nothing happens > on the SOLR side. If SOLR is not able to process, isn't it we get error from > SOLR instead of giving 200 OK response. > > Anybody has faced this sort of issue or any sort of help would be very much > appreciated. > > > > > On Sun, Sep 4, 2016 at 12:59 PM Ganesh M > <mgane...@live.in<mailto:mgane...@live.in>> wrote: > Nitin, Thanks for reply. Our each document has unique id and its hbase rowkey > id. So it will be unique only. So there is no chance of duplicates id being > send. > > > > On Sun 4 Sep, 2016 12:41 pm Nitin Kumar, > <nitinkumar.i...@gmail.com<mailto:nitinkumar.i...@gmail.com>> wrote: > Please check doc's unique key(Id). All keys shd be unique. Else docs having > same id will be replaced. > > On 04-Sep-2016 12:13 PM, "Ganesh M" > <mgane...@live.in<mailto:mgane...@live.in>> wrote: > >> Hi, >> we are keep sending documents to Solr from our app server. Single document >> per request, but in parallel of 10 request hits solr cloud in a second. >> >> We could see our post request ( update request ) hitting our solr 5.4 in >> localhost_access logs, and it's response as 200 Ok response. And also we >> get HTTP 200 OK response to our app servers as well for out HTTP request we >> fired to SOLR Cloud. >> >> But few documents are not getting indexed. Out of 2000 documents we sent >> 10 documents are getting missed. Thought there is not error, few documents >> are getting missed. >> >> We use autoSoftcommit as 2 secs and autohardcommit as 30 secs. >> >> Why is that 10 documents not getting indexed and also no error getting >> thrown back if server is not able to index it ? >> >> Regards, >> >> >> >>
