Hi David, given the fact that you are actually building a new index from scratch, my shot in the dark didn't hit any target. When you say : "Once the import finishes we save the docker image in the AWS docker repository. We then build our cluster using that image as the base"
Do you mean just configuraiton wise ? Will the new cluster have any starting index on disk? If i understood correctly your latest statements I expect a NO in here. So you are building a completely new index and comparing to the old index ( which is completely separate) you denote such a big difference in size. This is extremely suspicious . Optimizing in the end is just a huge merge to force 1 ( or N) final segments. Given the additional information you gave me, it's not going to make much difference. I would recommend to check how the index space is divided in different file formats [1] ( i.e. list how much space is dedicated to a specific extension) Stored content is in the .fdt files. [1] https://lucene.apache.org/core/6_4_0/core/org/apache/lucene/codecs/lucene62/package-summary.html#file-names ----- --------------- Alessandro Benedetti Search Consultant, R&D Software Engineer, Director Sease Ltd. - www.sease.io -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
