On Wed, 2015-10-14 at 10:17 +0200, Jan Høydahl wrote: > I have not benchmarked various number of segments at different sizes > on different HW etc, so my hunch could very well be wrong for Salman’s case. > I don’t know how frequent updates there is to his data either. > > Have you done #segments benchmarking for your huge datasets?
Only informally. However, the guys at UKWA run a similar scale index and have done multiple segment-count-oriented tests. They have not published a report, but there are measurements & graphs at https://github.com/ukwa/shine/tree/master/python/test-logs - Toke Eskildsen, State and University Library, Denmark
