Nawab Zada Asad Iqbal <khi...@gmail.com> wrote: > @Toke, I stumbled upon your page last week but it seems that your huge > index doesn't receive a lot of query traffic.
It switches between two kinds of usage: Everyday use is very low traffic by researchers using it interactively: 1-2 simultaneous queries, with faceting ranging from somewhat heavy to very heavy. Our setup is optimized towards this scenario and latency starts to go up pretty quickly if the number of simultaneous request rises. Now and then some cultural probes are being performed, where the index is being hammered continuously by multiple threads. Here it is our experience that max throughput for extremely simple queries (existence checks for social security numbers) is around 50 queries/second. > Mine is around 60TB and receives around 120 queries per second; ~90 shards on > 30 machines. Sounds interesting. Do you have a more detailed write-up somewhere? - Toke
