I'm running my tests on server with 4 double-kernel CPU. I was expecting good
improvements from multithreaded solution but I have speed 10th times worse.
Here is how I run those threads, I think I'm doing something wrong, please
advise:
------------------------------------------
............. code truncated .............
public class MultiFacetRequestHandler extends StandardRequestHandler {
protected NamedList getFacetInfo(SolrQueryRequest req,
SolrQueryResponse rsp,
DocSet mainSet) {
SimpleFacets f = new SimpleFacets(req.getSearcher(),
mainSet,
req.getParams());
NamedList facetInfo = f.getFacetCounts();
////////////////// This is custom code for multi facets
SolrParams p = req.getParams();
String fl = p.get(SolrParams.FL);
int flags = 0;
if (fl != null)
flags |= SolrPluginUtils.setReturnFields(fl, rsp);
Query query = QueryParsing.parseQuery(p.required().get(SolrParams.Q),
p.get(SolrParams.DF), p, req.getSchema());
try {
NamedList facetFields = (NamedList)
facetInfo.get("facet_fields");
if (facetFields.size() == 2) {
String shortFldName = facetFields.getName(0);
NamedList shortFld = (NamedList) facetFields.getVal(0);
NamedList longFld = (NamedList) facetFields.getVal(1);
if (shortFld.size() > longFld.size()) {
shortFld = longFld;
shortFldName = facetFields.getName(1);
}
List<Query> filters =
SolrPluginUtils.parseFilterQueries(req);
if (filters == null) filters = new LinkedList<Query>();
SolrIndexSearcher s = req.getSearcher();
Vector<Thread> threads = new Vector<Thread>();
Thread thread;
for (int i = 0; i < shortFld.size(); i++) {
SolrQueryParser qp = new SolrQueryParser(s.getSchema(),
null);
Query q = qp.parse(shortFldName + ":\"" +
shortFld.getName(i)+"\"");
List<Query> fltrs=new LinkedList<Query>();
fltrs.addAll(filters);
fltrs.add(q);
thread = new
Thread(makeRunnable(s,query,fltrs,flags,p,shortFld.getName(i),facetFields));
threads.add(thread);
thread.start();
}
for (Thread thread1 : threads) {
thread1.join();
}
}
} catch (Exception e) {
SolrException.logOnce(SolrCore.log, "Exception in multi faceting",
e);
}
///////////////////////////////////////////////
return facetInfo;
}
public Runnable makeRunnable(final SolrIndexSearcher s, final Query query,
final List<Query> filters, final int flags, final SolrParams p, final String
shrtName, final NamedList facetFields) {
return new Runnable() {
public void run() {
try{
DocListAndSet matrixRes = s.getDocListAndSet(query,
filters, null, 0, 0, flags);
NamedList matr = new
SimpleFacets(s,matrixRes.docSet,p).getFacetCounts();
facetFields.add(shrtName, matr.get("facet_fields"));
}catch (Exception e){
SolrException.logOnce(SolrCore.log, "Exception in multi
faceting", e);
}
}
};
}
............. code truncated .............
}
----- Original Message ----
From: Chris Hostetter <[EMAIL PROTECTED]>
To: [email protected]
Sent: Tuesday, February 26, 2008 2:55:36 AM
Subject: Re: Threads in Solr
: Yes I do computing the same DocSet. Should it be the problem? Is any way to
solve it?
: In general in each thread I ran the same query and add different Filter
Query.
it's not neccessarily a problem, it's just that you may not get much
benefit from prallelization if all of the worker threads are doing the
same work simulteneously.
but like i said: without knowing exactly what your threading code looks
like, it's hard to guess what might be wrong (and even if i was looking
right at your multithreaded code, it wouldn't neccessarily be obvious to
me, my multi-threading knowledge is mediocre) and it's still not clear if
you are testing on hardware that can actually take advantage of
parallelization.
-Hoss
