Using Solr 7.7.3-snapshot, 1 shard + 3 replicas on source and target cluster
When unidirectional CDCR enabled and buffer disabled, my understanding is, when data is successfully forwarded to target and committed, tlogs on both source and target should be purged. However, the source node doesn't purge tlogs no matter how I tried(manually committed as well) while tlogs on target are purged. (if I turn off CDCR and import data, tlogs is nicely cleaned) So I tested with some queries.. and there are no errors. queue size is 0, and the last processed version is not -1 either. I also double-checked CDCR buffer disabled on both source and target, and CDCR(unidirectioanl) data replication is working fine(except that fact that tlogs keep growing). What am I missing and what else should I check next? $ curl -k https://localhost:8983/solr/tbh_manuals_uni_shard1_replica_n2/cdcr?action=QUEUES { "responseHeader":{ "status":0, "QTime":0}, "queues":[ "host1:8981,host2:8981,host3:8981/solr",[ "tbh_manuals_uni",[ "queueSize",0, "lastTimestamp","2020-01-08T23:16:26.899Z"]]], "tlogTotalSize":503573, "tlogTotalCount":278, "updateLogSynchronizer":"stopped"} $ curl -k https://localhost:8983/solr/tbh_manuals_uni_shard1_replica_n2/cdcr?action=ERRORS { "responseHeader":{ "status":0, "QTime":1}, "errors":[ "host1:8981,host2:8981,host3:8981/solr",[ "tbh_manuals_uni",[ "consecutiveErrors",0, "bad_request",0, "internal",0, "last",[]]]]} $ curl -k https://localhost:8983/solr/tbh_manuals_uni_shard1_replica_n2/cdcr?action=LASTPROCESSEDVERSION { "responseHeader":{ "status":0, "QTime":0}, "lastProcessedVersion":1655203836093005824} I actually see some errors on zookeeper.out file only in target's leader node as follows. However honestly, I don't know what they mean.. 2020-01-08 15:11:42,740 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590008 type:create cxid:0xd2 zxid:0x3000000b4 txntype:-1 reqpath:n/a Error Path:/solr/collections Error:KeeperErrorCode = NodeExists for /solr/collections 2020-01-08 15:11:42,742 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590008 type:create cxid:0xd3 zxid:0x3000000b5 txntype:-1 reqpath:n/a Error Path:/solr/collections/tbh_manuals_uni Error:KeeperErrorCode = NodeExists for /solr/collections/tbh_manuals_uni 2020-01-08 15:11:42,744 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590008 type:create cxid:0xd4 zxid:0x3000000b6 txntype:-1 reqpath:n/a Error Path:/solr/collections/tbh_manuals_uni/terms Error:KeeperErrorCode = NodeExists for /solr/collections/tbh_manuals_uni/terms 2020-01-08 15:11:42,745 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590008 type:create cxid:0xd5 zxid:0x3000000b7 txntype:-1 reqpath:n/a Error Path:/solr/collections/tbh_manuals_uni/terms/shard1 Error:KeeperErrorCode = NodeExists for /solr/collections/tbh_manuals_uni/terms/shard1 2020-01-08 15:11:42,821 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590005 type:create cxid:0x23c zxid:0x3000000ba txntype:-1 reqpath:n/a Error Path:/solr/collections Error:KeeperErrorCode = NodeExists for /solr/collections 2020-01-08 15:11:42,823 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590005 type:create cxid:0x23d zxid:0x3000000bb txntype:-1 reqpath:n/a Error Path:/solr/collections/tbh_manuals_uni Error:KeeperErrorCode = NodeExists for /solr/collections/tbh_manuals_uni 2020-01-08 15:11:42,825 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590005 type:create cxid:0x23e zxid:0x3000000bc txntype:-1 reqpath:n/a Error Path:/solr/collections/tbh_manuals_uni/terms Error:KeeperErrorCode = NodeExists for /solr/collections/tbh_manuals_uni/terms 2020-01-08 15:11:42,827 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590005 type:create cxid:0x23f zxid:0x3000000bd txntype:-1 reqpath:n/a Error Path:/solr/collections/tbh_manuals_uni/terms/shard1 Error:KeeperErrorCode = NodeExists for /solr/collections/tbh_manuals_uni/terms/shard1 2020-01-08 15:11:45,185 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590005 type:setData cxid:0x274 zxid:0x3000000ce txntype:-1 reqpath:n/a Error Path:/solr/overseer/collection-queue-work/qnr-0000000016 Error:KeeperErrorCode = NoNode for /solr/overseer/collection-queue-work/qnr-0000000016 2020-01-08 15:11:45,187 [myid:2] - INFO [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x301d2ecaf590005 type:delete cxid:0x275 zxid:0x3000000cf txntype:-1 reqpath:n/a Error Path:/solr/overseer/collection-queue-work/qn-0000000016 Error:KeeperErrorCode = NoNode for /solr/overseer/collection-queue-work/qn-0000000016 -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
