This is very strange. I have no idea why DIH is taking so long. What happens if you execute the DIH query in some SQL front end? It's possible that DIH is just taking forever to execute the SQL query because of how it's written.
I'm having trouble following the query results, but again this is super-slow. How many docs do you have in your index? How much memory have you allocated for your JVM? Your query is actually pretty simple, so I have no clue whatsoever why your response times are so slow. But this is such bad performance that something major is wrong. I'd approach it one problem at a time. Understand what's happening with your queries, _then_ tackle DIH (or the other way around)... Best, Erick On Wed, Jul 16, 2014 at 2:03 AM, YouPeng Yang <yypvsxf19870...@gmail.com> wrote: > Hi > I build my SolrCloud using Solr 4.6.0 (java version:1.7.0_45). In my > cloud,I have a collection with 30 shard,and each shard has one replica. > each core of the shard contains nearly 50 million docs that is 15GB in > size,so does the replica. > Before applying my cloud in the real world,I do a performance test with > JMeter 2.11. > The scenario of the my test is simple:100 threads sending requests for 20 > seconds ,and these requests are only sent to a specific core of a specific > shard.the request is similar to the following : > http://IP:port/solr/tv_201407/select?q=*:*&fq=BEGINTIME:["2014-06-01 > 00:00:00"+TO+*]+AND+(CONTACT:${user})+AND (TV_STATE:"00")&shards=tv_201407 > &rows=2000&sort=BEGINTIME+desc. > > I encountered the drastic query latency during performance testing and > DIH import delay after performance testing.Please help me. I have tested > several times and get the same problem and can not handle it by myself.Any > suggestion will be apprecaited. > > The following steps describes what I have done . > > Step 1: Before the test,the DIH import job is very fast.As the statistics > [1], the DIH importing takes only 1s for 10 docs. > [1]----------------------------------- > Indexing completed. Added/Updated: 10 documents. Deleted 0 documents. > (Duration: 01s) > Requests: 1 (1/s), Fetched: 10 (10/s), Skipped: 0, Processed: 10 (10/s) > Started: less than a minute ago > --------------------------------------- > > Step 2: Then ,Doing the test under the caches are cleaned. The summery > statistics data is as [2]. Although I have clean the caches,I never think > the query latency becomes so drastic that it cannot be acceptable in my > real application. > The red font describes the latency of the query performance test on the > core tv_201407 of the shard tv_201407 . > > So would you experts can give some hints about the drastic query latency > ? > > [2]----------------------------------- > [solr@solr2 test]$ ../bin/jmeter.sh -n -t solrCoudKala20140401.jmx -l > logfile_solrCloud_20.jtl > Creating summariser <aggregate> > Created the tree successfully using solrCoudKala20140401.jmx > Starting the test @ Wed Jul 16 15:59:28 CST 2014 (1405497568104) > Waiting for possible shutdown message on port 4445 > aggregate + 1 in 8.1s = 0.1/s Avg: 8070 Min: 8070 Max: 8070 > Err: > 0 (0.00%) Active: 100 Started: 100 Finished: 0 > aggregate + 103 in 13.4s = 7.7/s Avg: 8027 Min: 4191 Max: 8434 > Err: > 0 (0.00%) Active: 97 Started: 100 Finished: 3 > aggregate = 104 in 13.4s = 7.7/s Avg: 8027 Min: 4191 Max: 8434 > Err: > 0 (0.00%) > aggregate + 96 in 7s = 14.5/s Avg: 6160 Min: 5295 Max: 6625 > Err: > 0 (0.00%) Active: 0 Started: 100 Finished: 100 > aggregate = 200 in 15s = 13.6/s Avg: 7131 Min: 4191 Max: 8434 > Err: > 0 (0.00%) > Tidying up ... @ Wed Jul 16 15:59:43 CST 2014 (1405497583461) > ... end of run > [solr@solr2 test]$ > --------------------------------------- > Step 3:To be continued,after the test,I do the DIH importing job again > using the same import expresion.However the performance of the DIH becomes > so unacceptable. > to import the 10 docs takes 2 m 15 s [3]! > Having noticing that ,solr can fetched the 10 docs fast,the processing is > slow. > > [3]----------------------------------- > *Indexing completed. Added/Updated: 10 documents. Deleted 0 documents. > (Duration: 2m 15s)* > Requests: 1 (0/s), Fetched: 10 (0/s), Skipped: 0, Processed: 10 (0/s) > Started: about an hour ago > --------------------------------------- > > By the way. jvm gc goes normal,and there is no long full gc during the > test. the load of my system(rhel 6.5) are also normal. > > Regards >
