Re: Re: Low untunable default FastWriter output buffer - possible reason for slow single threaded data receiving from Solr on 1Gigabit+ networks while scroll, search etc

2023-03-14 Thread David Smiley
You can get parallelism by sharding -- use numShards=8 or whatever number of CPUs you have. In your performance analysis, you did not speak of memory; you spoke as if there are only two factors at play -- CPU and Disk. Compression of data on disk is used in order to allow the OS's disk cache to c

RE: Re: Low untunable default FastWriter output buffer - possible reason for slow single threaded data receiving from Solr on 1Gigabit+ networks while scroll, search etc

2023-03-14 Thread Fikavec F
Thank you for working on the Solr performance issues raised here.   LZ4 is a great solution, but let's look at how things are today. As far as I understand, uncompressed fields have been abandoned since version 4.1 (early 2013). At that time, 15,000 RPM SAS disks produced 350 MB/s or 150-180 iops a