I found the problem's cause.It's the DocSetCollector. my fitler query result's
size is about 3000000,so the DocSetCollector.getDocSet() is OpenBitSet. And
3000000 OpenBitSet.fastSet(doc) op is too slow. So I used SolrIndexSearcher's
TopFieldDocs search(Query query, Filter filter, int n,
Sort sort), and it's normal.
At 2010-10-20 19:21:27，kafka0102 <[email protected]> wrote:
>For solr's SolrIndexSearcher.search(QueryResult qr, QueryCommand cmd), I find
>it's too slowly.my index's size is about 500M, and record's num is 3984274.my
>query is like q=xx&fq=fid:1&fq=atm:[int_time1 TO int_time2].
>fid's type is <fieldType name="int" class="solr.TrieIntField"
>precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
>atm's type is <fieldType name="sint" class="solr.TrieIntField"
>precisionStep="8" omitNorms="true" positionIncrementGap="0"/>.
>for the test, I closed solr's cache's config and used another lucene's code
>like bottom:
>
> private void test2(final ResponseBuilder rb) {
> try {
> final SolrQueryRequest req = rb.req;
> final SolrIndexSearcher searcher = req.getSearcher();
> final SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
> final ExecuteTimeStatics timeStatics =
> ExecuteTimeStatics.getExecuteTimeStatics();
> final ExecuteTimeUnit staticUnit =
> timeStatics.addExecuteTimeUnit("test2");
> staticUnit.start();
> final List<Query> query = cmd.getFilterList();
> final BooleanQuery booleanFilter = new BooleanQuery();
> for (final Query q : query) {
> booleanFilter.add(new BooleanClause(q,Occur.MUST));
> }
> booleanFilter.add(new BooleanClause(cmd.getQuery(),Occur.MUST));
> logger.info("q:"+query);
> final Sort sort = cmd.getSort();
> final TopFieldDocs docs = searcher.search(booleanFilter,null,20,sort);
> final StringBuilder sbBuilder = new StringBuilder();
> for (final ScoreDoc doc :docs.scoreDocs) {
> sbBuilder.append(doc.doc+",");
> }
> logger.info("hits:"+docs.totalHits+",result:"+sbBuilder.toString());
> staticUnit.end();
> } catch (final Exception e) {
> throw new RuntimeException(e);
> }
> }
>
>for the test, I first called above's code and then solr's search(...). The
>result is : lucence's about 20ms and solr's about 70ms.
>I'm so confused.
>And,I wrote another code using filter like bottom,but the range query's result
>num is not correct.
>Can anybody knows the reasons?
>
> private void test1(final ResponseBuilder rb) {
> try {
> final SolrQueryRequest req = rb.req;
> final SolrIndexSearcher searcher = req.getSearcher();
> final SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
> final ExecuteTimeStatics timeStatics =
> ExecuteTimeStatics.getExecuteTimeStatics();
> final ExecuteTimeUnit staticUnit =
> timeStatics.addExecuteTimeUnit("test1");
> staticUnit.start();
> final List<Query> query = cmd.getFilterList();
> final BooleanFilter booleanFilter = new BooleanFilter();
> for (final Query q : query) {
> setFilter(booleanFilter,q);
> }
> final Sort sort = cmd.getSort();
> final TopFieldDocs docs =
> searcher.search(cmd.getQuery(),booleanFilter,20,sort);
> logger.info("hits:"+docs.totalHits);
>
> staticUnit.end();
> } catch (final Exception e) {
> throw new RuntimeException(e);
> }
> }
>
>
