Hello! Another question I could not find an answer to: is there a best-practice / recommendation for pushing several million documents into a new index?
I'm currently splittig my documents into batches of 10,000 json-line payloads into the update request handler, with commit set to 'true' (yes, for each of the batches). I'm using commit since that got me stable 'QTime' around ~2100, without commiting every batch, the QTime will degrade ten-fold by the time I sent somewhere around 1,000,000 documents. This will steadily climb, so after I sent all 46M documents I end up with QTime values about 40,000 in case I don't commit every batch immediately. Since I cannot find anything in my mails, I wanted to search the solr-user archives but, as far as I can tell: there is no such thing. Maybe I can't see it or just glossed over it, but is there no searchable index of solr-user? Any hints? regards, -1
