hi all
I want to speed up search time for my application. In a query, the
time is largly used in reading postlist(io with frq files) and
calculate scores and collect result(cpu, with Priority Queue). IO is
hardly optimized or already part optimized by nio. So I want to use
multithreads to utilize cpu. of course, it may be decrease QPS, but
the response time will also decrease-- that what I want. Because cpu
is easily obtained compared to faster hard disk.
I read the codes of searching roughly and find it's not an easy
task to modify search process. So I want to use other easy method .
One is use solr distributed search and dispatch documents to many
shards. but due to the network and global idf problem,it seems not a
good method for me.
Another one is to modify the index structure and averagely
dispatch frq files.
e.g term1 -> doc1,doc2, doc3,doc4,doc5 in _1.frq
I create to 2 indexes with
term1->doc1,doc3,doc5
term1->doc2,doc4
when searching, I create 2 threads with 2 PriorityQueues to
collect top N docs and merging their results
Is the 2nd idea feasible? Or any one has related idea? thanks.
