Solr5.X document loss in splitting shards

[Luca Quarello]

Hi,
I have a SOLR 5.3.1 CLOUD with two nodes and 8 shards per node.

Each shard is about* 35 million documents (**35025882**) and 16GB sized.*


   - I launch the SPLIT command on a shard (shard 13) in the ASYNC way:

curl "
http://x-perf-jvm5:8983/solr/admin/collections?action=SPLITSHARD&collection=sepa&shard=shard13&async=1006
"



   - After many time I obtain:


curl "
http://x-perf-jvm5:8983/solr/admin/collections?action=REQUESTSTATUS&requestid=1006
"

<response>
<lst name="responseHeader"><int name="status">0</int><int
name="QTime">6</int></lst><lst name="success"><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">2</int></lst></lst><lst><lst name="responseHeader"><int
name="status">0</int><int name="QTime">0</int></lst></lst><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst></lst><lst><lst name="responseHeader"><int
name="status">0</int><int name="QTime">0</int></lst></lst><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst></lst><lst><lst name="responseHeader"><int
name="status">0</int><int name="QTime">5</int></lst></lst><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">5</int></lst><str
name="core">sepa_shard13_1_replica1</str><str
name="status">EMPTY_BUFFER</str></lst><lst name="
http://192.168.72.55:8983/solr/sepa_shard13_replica1/";><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">481</int></lst></lst></lst><lst
name="1006264805140687740"><lst name="responseHeader"><int
name="status">0</int><int name="QTime">0</int></lst><str
name="STATUS">completed</str><str name="Response">TaskId:
1006264805140687740 webapp=null path=/admin/cores
params={shard=shard13_0&amp;collection.configName=flsFragments&amp;name=sepa_shard13_0_replica1&amp;action=CREATE&amp;collection=sepa&amp;wt=javabin&amp;qt=/admin/cores&amp;async=1006264805140687740&amp;version=2}
status=0 QTime=2 </str></lst><lst name="1006264808287598167"><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst><str name="STATUS">completed</str><str
name="Response">TaskId: 1006264808287598167 webapp=null path=/admin/cores
params={shard=shard13_1&amp;collection.configName=flsFragments&amp;name=sepa_shard13_1_replica1&amp;action=CREATE&amp;collection=sepa&amp;wt=javabin&amp;qt=/admin/cores&amp;async=1006264808287598167&amp;version=2}
status=0 QTime=0 </str></lst><lst name="1006264810307413066"><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst><str name="STATUS">completed</str><str
name="Response">TaskId: 1006264810307413066 webapp=null path=/admin/cores
params={coreNodeName=core_node18&amp;state=active&amp;nodeName=192.168.72.55:8983_solr&amp;action=PREPRECOVERY&amp;checkLive=true&amp;core=sepa_shard13_1_replica1&amp;wt=javabin&amp;qt=/admin/cores&amp;onlyIfLeader=true&amp;async=1006264810307413066&amp;version=2}
status=0 QTime=0 </str></lst><lst name="1006264810317508052"><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst><str name="STATUS">completed</str><str
name="Response">TaskId: 1006264810317508052 webapp=null path=/admin/cores
params={targetCore=sepa_shard13_0_replica1&amp;targetCore=sepa_shard13_1_replica1&amp;action=SPLIT&amp;core=sepa_shard13_replica1&amp;wt=javabin&amp;qt=/admin/cores&amp;async=1006264810317508052&amp;version=2}
status=0 QTime=0 </str></lst><lst name="1006266054432757899"><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">0</int></lst><str name="STATUS">completed</str><str
name="Response">TaskId: 1006266054432757899 webapp=null path=/admin/cores
params={name=sepa_shard13_1_replica1&amp;action=REQUESTAPPLYUPDATES&amp;wt=javabin&amp;qt=/admin/cores&amp;async=1006266054432757899&amp;version=2}
status=0 QTime=5 </str></lst><lst name="status"><str
name="state">completed</str><str name="msg">found 1006 in completed
tasks</str></lst>
</response>



   - I launch the commit command:

curl http://x-perf-jvm5:8983/solr/sepa/update --data-binary '<commit/>' -H
'Content-type:application/xml'

<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int
name="QTime">162</int></lst>
</response>



The new created shards have:
*13430316 documents (5.6 GB) and 13425924 documents (5.59 GB**)*.

What is the problem? Where I am wrong?

Thanks,
Luca