Re: Re: Re: Low untunable default FastWriter output buffer - possible reason for slow single threaded data receiving from Solr on 1Gigabit+ networks while scroll, search etc

As being said above, the speed of streaming file to socket, via OS internals is hardly achievable with java code crunching bytes through heap. Using RamDirectory might push on GC ,it's rather better to stick with the default one and leave enough RAM for file cache. Regarding the actual params: q=*

RE: Re: Re: Low untunable default FastWriter output buffer - possible reason for slow single threaded data receiving from Solr on 1Gigabit+ networks while scroll, search etc

Thanks for the patch for testing. I could not see significant improvements on virtual machines, I will try again this week on servers.I tried the following values for buffers: 65536 -  64Kb, 262144 - 256Kb, 524288 - 512Kb, 1048576 - 1MB, 4194304 - 4MB, 16777216 - 16MB, 33554432 - 32Mb, 67108864 - 6

RE: Re: Low untunable default FastWriter output buffer - possible reason for slow single threaded data receiving from Solr on 1Gigabit+ networks while scroll, search etc

Today I created a 1GB collection with RAMDirectoryFactory and only id, stored unindexed without docValues text_sn fields in the schema. In solrconfig.xml:... and in section:${solr.lock.type:single} In managed-schema:...        ... Receiving all data from it is also carried out at slow speeds of ap

Re: Re: Low untunable default FastWriter output buffer - possible reason for slow single threaded data receiving from Solr on 1Gigabit+ networks while scroll, search etc

Hi, I made it tunable. It should capture it from OS env BUFSIZE=8192, and java property -DBUFSIZE=8192. It logs actual value under INFO for FastWriter https://github.com/mkhludnev/lucene-solr/commit/f0ed425bcbb16c50d01f3ff7fba3879148f50568 Here's a jar https://github.com/mkhludnev/lucene-solr/relea