Re: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

Hi Danilo, following my experience now SSD or RAM Disk is the only way to speed up queries. It depends on your storage occupation of your 41M docs. If you don't have Enterprise SSD you can add consumer SSD as a fast cache (linux caching modules "flashcache / bcache" are able to use cheap SSD as a d

Query regarding integrating solr query functions into blockfacetjoin Query

Hi Team, I was implementing block join faceting query in my project and was stuck in integrating the existing functional queries in the block join faceting query. *The current query using 'select' handler is as follows* :- https://localhost:8983/solr/master_Product_default/*select*?*yq* =_query_:

Re: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

On 2021-02-22 1:52 AM, Danilo Tomasoni wrote: Hello all, we are running a solr instance with around 41 MLN documents on a SATA class 10 disk with around 10.000 rpm. We are experiencing very slow query responses (in the order of hours..) with an average of 205 segments. We made a test with a nor

Re: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

A forced merge might improve speed 20%. Going from spinning disk to SSD will improve speed 20X or more. Don’t waste your time even thinking about forced merges. You need to get SSDs. The even bigger speedup is to get enough RAM that the OS can keep the Solr index files in file system buffers. Ch

Re: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

On 2/22/2021 12:52 AM, Danilo Tomasoni wrote: we are running a solr instance with around 41 MLN documents on a SATA class 10 disk with around 10.000 rpm. We are experiencing very slow query responses (in the order of hours..) with an average of 205 segments. We made a test with a normal pc and

Re: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

On 2021-02-22 11:18 AM, Shawn Heisey wrote: The OS automatically uses unallocated memory to cache data on the disk.  Because memory is far faster than any disk, even SSD, it performs better. Depends on the os, from "defragmenting solrdata folder" I suspect the OP is on windows whose filesyst

Re: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

True, but Windows does cache files. It has been a couple of decades since I ran search on Windows, but Ultraseek got large gains from setting some sort of system property to make it act like a file server and give file caching equal priority with program caching. wunder Walter Underwood wun...@

Caffeine Cache and Filter Cache in 8.3

Hi SOLR Community, I have a question about cache implementations based on some seemingly inconsistent documentation I'm looking at. I'm currently inquiring about 8.3, but more generally about solr version 8 too for upgrade planning. In the description in the docs for cache implementations says

Re: Caffeine Cache and Filter Cache in 8.3

On 2/22/2021 1:50 PM, Stephen Lewis Bianamara wrote: (a) At what version did the caffeine cache reach production stability? (b) Is the caffeine cache, and really all implementations, able to be used on any cache, or are the restrictions about which cache implementations may be used for which c

Re: Is 8.8.x going be stabilized and finalized?

Hey Subhajit, Can you share briefly what issues are being seen with 8.7+ versions? We are planning to move a big workload from 7.6 to 8.7 version. We created a small load-testing tool for sanitizing new Solr versions and that showed throughput of traffic decreasing much more than Solr 7.6 as we l

R: defragmentation can improve performance on SATA class 10 disk ~10000 rpm ?

Thank you all for the suggestions, The OS is not windows, it's centos, a colleague thinks that even on linux defragmenting can improve performance about 2X because it keeps the data contiguous on disk. We cannot use flashcache because we run solr on virtual machines. We will investigate better o